0 ratings0% found this document useful (0 votes)
14 views1,799 pagesLog4j Active
The document contains log entries from a system indicating various operations and statuses, including errors, warnings, and informational messages related to a Python driver and file system configurations. It highlights issues with environment variables, deprecated settings, and the progress of tasks within a Spark job. Additionally, it reports on the completion of tasks, job statuses, and permissions checks within a Databricks environment.
Uploaded by
mipipel212Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
0 ratings0% found this document useful (0 votes)
14 views1,799 pagesLog4j Active
The document contains log entries from a system indicating various operations and statuses, including errors, warnings, and informational messages related to a Python driver and file system configurations. It highlights issues with environment variables, deprecated settings, and the progress of tasks within a Spark job. Additionally, it reports on the completion of tasks, job statuses, and permissions checks within a Databricks environment.
Uploaded by
mipipel212Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 1799
25/02/06 06:00:03 ERROR DriverCorral: Could not get repl, python repl for context
4543109989918105790 is not ready or in a bad state.
25/02/06 06:00:05 INFO PythonDriverWrapper: setupRepl:ReplId-4af0e-2c437-37ac6-6:
finished to load
25/02/06 06:00:06 INFO DriverConf: Configured feature flag data source LaunchDarkly
25/02/06 06:00:06 WARN DriverConf: REGION environment variable is not defined.
getConfForCurrentRegion will always return default value
25/02/06 06:00:06 INFO ProgressReporter$: Reporting progress for running commands:
5400065285585087590_6933734711743617059_8c222973c44c4427a9a4a68ed05a8ec7
25/02/06 06:00:06 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_6933734711743617059_8c222973c44c4427a9a4a68ed05a8ec7
25/02/06 06:00:07 INFO ProgressReporter$: Reporting partial results for running
commands: 5400065285585087590_6933734711743617059_8c222973c44c4427a9a4a68ed05a8ec7
25/02/06 06:00:09 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_6933734711743617059_8c222973c44c4427a9a4a68ed05a8ec7
25/02/06 06:00:09 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_6507497350060734899_c29d4a497993433cb6d8b5130b2caa62
25/02/06 06:00:09 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_6507497350060734899_c29d4a497993433cb6d8b5130b2caa62
25/02/06 06:00:09 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_7198893288194829193_bbb7015911ed4764bf33b147d896deca
25/02/06 06:00:09 INFO deprecation: fs.default.name is deprecated. Instead, use
fs.defaultFS
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme s3n. Current value: com.databricks.sql.acl.fs.CredentialScopeFileSystem.
Old value to be restored: com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme s3a. Current value: com.databricks.sql.acl.fs.CredentialScopeFileSystem.
Old value to be restored: com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme abfss. Current value:
com.databricks.sql.acl.fs.CredentialScopeFileSystem. Old value to be restored:
com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme gs. Current value: com.databricks.sql.acl.fs.CredentialScopeFileSystem.
Old value to be restored: com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme s3. Current value: com.databricks.sql.acl.fs.CredentialScopeFileSystem.
Old value to be restored: com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO HadoopFSUtil$: Installing the old filesystem implementation
for scheme abfs. Current value:
com.databricks.sql.acl.fs.CredentialScopeFileSystem. Old value to be restored:
com.databricks.common.filesystem.LokiFileSystem.
25/02/06 06:00:09 INFO DatabricksFileSystemV2Factory: Creating S3A file system for
s3a://o9-gmg-datalake-prod-mena-eu-1
25/02/06 06:00:11 WARN DatabricksS3LoggingUtils$:V3: Error calling HeadBucket: 400.
This is typically used to infer the region of the bucket and is harmless. Details:
Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: i-
00f4ec534ef0e57a4 Response Headers: Map(Transfer-Encoding -> chunked, Server ->
AmazonS3, x-amz-id-2 ->
JkC1Uy2p3bpCrfkJXirK+yfu4oQtzV+zSXJFi8+5GBEwa1NVyJiYHy8Kr3DuY1niuI5aAbCESak=,
Connection -> close, x-amz-bucket-region -> eu-west-1, Content-Type ->
application/xml, x-amz-request-id -> ZANTTWJEMMV22V8W, Date -> Thu, 06 Feb 2025
06:00:10 GMT)
25/02/06 06:00:11 INFO deprecation: fs.default.name is deprecated. Instead, use
fs.defaultFS
25/02/06 06:00:11 WARN DatabricksS3LoggingUtils$:V3: Error calling HeadBucket: 400.
This is typically used to infer the region of the bucket and is harmless. Details:
Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: i-
00f4ec534ef0e57a4 Response Headers: Map(Transfer-Encoding -> chunked, Server ->
AmazonS3, x-amz-id-2 ->
s84bmtmmh+sS98BYFC53405BzWcNiAvM5r5N/v1eQNbU0CEa4bR5MufonH62ERYKf8clbRfiE9Y=,
Connection -> close, x-amz-bucket-region -> eu-west-1, Content-Type ->
application/xml, x-amz-request-id -> N1J24H33PSZAFZ5Q, Date -> Thu, 06 Feb 2025
06:00:11 GMT)
25/02/06 06:00:12 INFO InMemoryFileIndex: Start listing leaf files and directories.
Size of Paths: 1; threshold: 32
25/02/06 06:00:12 INFO InMemoryFileIndex: Start listing leaf files and directories.
Size of Paths: 0; threshold: 32
25/02/06 06:00:12 INFO InMemoryFileIndex: It took 156 ms to list leaf files for 1
paths.
25/02/06 06:00:13 INFO MemoryStore: Block broadcast_0 stored as values in memory
(estimated size 156.4 KiB, free 20.2 GiB)
25/02/06 06:00:13 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in
memory (estimated size 18.4 KiB, free 20.2 GiB)
25/02/06 06:00:13 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.232.44:37705 (size: 18.4 KiB, free: 20.2 GiB)
25/02/06 06:00:13 INFO SparkContext: Created broadcast 0 from writeExternal at
ObjectOutputStream.java:1459
25/02/06 06:00:13 INFO SparkContext: Starting job: parquet at
NativeMethodAccessorImpl.java:0
25/02/06 06:00:13 INFO DAGScheduler: Got job 0 (parquet at
NativeMethodAccessorImpl.java:0) with 1 output partitions
25/02/06 06:00:13 INFO DAGScheduler: Final stage: ResultStage 0 (parquet at
NativeMethodAccessorImpl.java:0)
25/02/06 06:00:13 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:00:13 INFO DAGScheduler: Missing parents: List()
25/02/06 06:00:13 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1]
at parquet at NativeMethodAccessorImpl.java:0), which has no missing parents
25/02/06 06:00:13 INFO DBCEventLoggingListener: Rolling event log;
numTimesRolledOver = 1
25/02/06 06:00:13 INFO DBCEventLoggingListener: Rolled active log file
/databricks/driver/eventlogs/9206972423021441211/eventlog to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-00, size =
22083815
25/02/06 06:00:13 INFO DBCEventLoggingListener: Logging events to
eventlogs/9206972423021441211/eventlog
25/02/06 06:00:13 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0
(MapPartitionsRDD[1] at parquet at NativeMethodAccessorImpl.java:0) (first 15 tasks
are for partitions Vector(0))
25/02/06 06:00:13 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks resource
profile 0
25/02/06 06:00:13 INFO TaskSetManager: TaskSet 0.0 using PreferredLocationsV1
25/02/06 06:00:13 WARN FairSchedulableBuilder: A job was submitted with scheduler
pool 5400065285585087590, which has not been configured. This can happen when the
file that pools are read from isn't set, or when that file doesn't contain
5400065285585087590. Created 5400065285585087590 with default configuration
(schedulingMode: FIFO, minShare: 0, weight: 1)
25/02/06 06:00:13 INFO FairSchedulableBuilder: Added task set TaskSet_0.0 tasks to
pool 5400065285585087590
25/02/06 06:00:13 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:00:14 INFO DBCEventLoggingListener: Compressed rolled file
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-00 to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-00.gz in
761ms, size = 3428009
25/02/06 06:00:14 INFO DBCEventLoggingListener: Deleted rolled file
eventlogs/9206972423021441211/eventlog-2025-02-06--06-00
25/02/06 06:00:15 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.232.45:45579 (size: 18.4 KiB, free: 18.4 GiB)
25/02/06 06:00:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:00:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:00:24 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in
10529 ms on 10.183.232.45 (executor 2) (1/1)
25/02/06 06:00:24 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:00:24 INFO DAGScheduler: ResultStage 0 (parquet at
NativeMethodAccessorImpl.java:0) finished in 10.780 s
25/02/06 06:00:24 INFO DAGScheduler: Job 0 is finished. Cancelling potential
speculative or zombie tasks for this job
25/02/06 06:00:24 INFO TaskSchedulerImpl: Killing all running tasks in stage 0:
Stage finished
25/02/06 06:00:24 INFO DAGScheduler: Job 0 finished: parquet at
NativeMethodAccessorImpl.java:0, took 10.966761 s
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.014141 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
12.840602 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
55.033328 milliseconds
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.093755 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
2.089562 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
38.418768 milliseconds
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.128521 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
1.458129 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
31.246773 milliseconds
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.103291 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
2.81427 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
32.062316 milliseconds
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.111955 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
1.522874 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
32.556577 milliseconds
25/02/06 06:00:24 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
0.21023 ms.
25/02/06 06:00:24 INFO HiveCheckPermissions: Computed permissions to check in
1.404991 milliseconds
25/02/06 06:00:24 INFO HiveCheckPermissions: Checking all permissions took
31.347911 milliseconds
25/02/06 06:00:24 INFO ClusterLoadMonitor: Added query with execution ID:31.
Current active queries:1
25/02/06 06:00:24 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 12, Total cores: 12, Current load: 1, Current Avg
load: 0, New parallelism: 12
25/02/06 06:00:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0,
New Ema: 1.0
25/02/06 06:00:25 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(channel_name),EqualTo(channel_name,CLEARANCE)
25/02/06 06:00:25 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(channel_name#2),(channel_name#2 = CLEARANCE)
25/02/06 06:00:26 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 4194304, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:00:27 INFO DAGScheduler: Registering RDD 6 (wrapper at /home/spark-
46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7) as
input to shuffle 0
25/02/06 06:00:27 INFO DAGScheduler: Got map stage job 1 (wrapper at /home/spark-
46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7)
with 1 output partitions
25/02/06 06:00:27 INFO DAGScheduler: Final stage: ShuffleMapStage 1 (wrapper at
/home/spark-46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-
776709266:7)
25/02/06 06:00:27 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:00:27 INFO DAGScheduler: Missing parents: List()
25/02/06 06:00:27 INFO DAGScheduler: Submitting ShuffleMapStage 1
(MapPartitionsRDD[6] at wrapper at /home/spark-46b4cba4-6e97-4786-9ef9-
da/.ipykernel/1822/command-589147838710964-776709266:7), which has no missing
parents
25/02/06 06:00:27 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 1 (MapPartitionsRDD[6] at wrapper at /home/spark-46b4cba4-6e97-
4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7) (first 15 tasks
are for partitions Vector(0))
25/02/06 06:00:27 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks resource
profile 0
25/02/06 06:00:27 INFO TaskSetManager: TaskSet 1.0 using PreferredLocationsV1
25/02/06 06:00:27 WARN FairSchedulableBuilder: A job was submitted with scheduler
pool 5400065285585087590, which has not been configured. This can happen when the
file that pools are read from isn't set, or when that file doesn't contain
5400065285585087590. Created 5400065285585087590 with default configuration
(schedulingMode: FIFO, minShare: 0, weight: 1)
25/02/06 06:00:27 INFO FairSchedulableBuilder: Added task set TaskSet_1.0 tasks to
pool 5400065285585087590
25/02/06 06:00:27 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:00:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:00:30 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in
3223 ms on 10.183.232.45 (executor 2) (1/1)
25/02/06 06:00:30 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:00:30 INFO DAGScheduler: ShuffleMapStage 1 (wrapper at /home/spark-
46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7)
finished in 3.336 s
25/02/06 06:00:30 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:00:30 INFO DAGScheduler: running: Set()
25/02/06 06:00:30 INFO DAGScheduler: waiting: Set()
25/02/06 06:00:30 INFO DAGScheduler: failed: Set()
25/02/06 06:00:30 INFO ShufflePartitionsUtil: For shuffle(0), advisory target size:
67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:00:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:00:31 INFO CodeGenerator: Code generated in 866.135892 ms
25/02/06 06:00:31 INFO SparkContext: Starting job: wrapper at /home/spark-46b4cba4-
6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7
25/02/06 06:00:31 INFO DAGScheduler: Got job 2 (wrapper at /home/spark-46b4cba4-
6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7) with 1
output partitions
25/02/06 06:00:31 INFO DAGScheduler: Final stage: ResultStage 3 (wrapper at
/home/spark-46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-
776709266:7)
25/02/06 06:00:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
2)
25/02/06 06:00:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:00:31 INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[10]
at wrapper at /home/spark-46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-
589147838710964-776709266:7), which has no missing parents
25/02/06 06:00:31 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 3
(MapPartitionsRDD[10] at wrapper at /home/spark-46b4cba4-6e97-4786-9ef9-
da/.ipykernel/1822/command-589147838710964-776709266:7) (first 15 tasks are for
partitions Vector(0))
25/02/06 06:00:31 INFO TaskSchedulerImpl: Adding task set 3.0 with 1 tasks resource
profile 0
25/02/06 06:00:31 INFO TaskSetManager: TaskSet 3.0 using PreferredLocationsV1
25/02/06 06:00:31 WARN FairSchedulableBuilder: A job was submitted with scheduler
pool 5400065285585087590, which has not been configured. This can happen when the
file that pools are read from isn't set, or when that file doesn't contain
5400065285585087590. Created 5400065285585087590 with default configuration
(schedulingMode: FIFO, minShare: 0, weight: 1)
25/02/06 06:00:31 INFO FairSchedulableBuilder: Added task set TaskSet_3.0 tasks to
pool 5400065285585087590
25/02/06 06:00:31 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 2)
(10.183.231.236, executor 4, partition 0, PROCESS_LOCAL,
25/02/06 06:00:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 0 to 10.183.231.236:58934
25/02/06 06:00:34 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:00:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:00:39 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 2) in
7235 ms on 10.183.231.236 (executor 4) (1/1)
25/02/06 06:00:39 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:00:39 INFO DAGScheduler: ResultStage 3 (wrapper at /home/spark-
46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7)
finished in 7.263 s
25/02/06 06:00:39 INFO DAGScheduler: Job 2 is finished. Cancelling potential
speculative or zombie tasks for this job
25/02/06 06:00:39 INFO TaskSchedulerImpl: Killing all running tasks in stage 3:
Stage finished
25/02/06 06:00:39 INFO DAGScheduler: Job 2 finished: wrapper at /home/spark-
46b4cba4-6e97-4786-9ef9-da/.ipykernel/1822/command-589147838710964-776709266:7,
took 7.282135 s
25/02/06 06:00:39 INFO ClusterLoadMonitor: Removed query with execution ID:31.
Current active queries:0
25/02/06 06:00:39 INFO DiskCache$: Initializing disk cache, paths =
[/local_disk0/dbio_cache_root_650-171e0999-923f-4bc7-b429-622ee1c6a82e/
unified_cache-38ddb417-4595-4b8a-b644-59577ba7c193], maxDiskUsage =
1073741824,guaranteedDiskUtilization = 0.0, minDiskAvailable = -1, policy =
SLRU_0.2, S3Fifo_0.2, S3Fifo+_0.2
25/02/06 06:00:39 INFO SQLAppStatusListener: Recording cache-related metrics in
usage logs:
ioCacheWorkersDiskUsage={"10.183.229.123":
{"diskUsage":0,"lifetime":251219},"10.183.232.45":
{"diskUsage":0,"lifetime":253187},"10.183.231.236":
{"diskUsage":0,"lifetime":250444}},
ioCacheNumScanTasks={"numLocalScanTasks":0,"numNonLocalScanTasks":1},
ioCacheDiskUsageLimit=1073741824
25/02/06 06:00:39 INFO SQLAppStatusListener: Recording DV cache-related metrics in
usage logs: {"10.183.229.123":
{"memCacheUsage":0,"diskCacheUsage":0,"lifetimeMs":251295},"10.183.232.45":
{"memCacheUsage":0,"diskCacheUsage":0,"lifetimeMs":253263},"10.183.231.236":
{"memCacheUsage":0,"diskCacheUsage":0,"lifetimeMs":250520}}
25/02/06 06:00:39 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_7198893288194829193_bbb7015911ed4764bf33b147d896deca
25/02/06 06:00:39 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_6445877654742662091_c734fdbf6f7f42e58c30d94b3433d3f5
25/02/06 06:00:39 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_6445877654742662091_c734fdbf6f7f42e58c30d94b3433d3f5
25/02/06 06:00:39 INFO QueryProfileListener: Query profile sent to logger, seq
number: 31, app id: app-20250206055605-0000
25/02/06 06:00:40 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0,
New Ema: 0.85
25/02/06 06:00:43 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema:
0.85, New Ema: 0.0
25/02/06 06:01:00 INFO DataSourceFactory$: DataSource Jdbc URL:
jdbc:mariadb://md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com:3306/
organization8517174019523704?
useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertif
icate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt
25/02/06 06:01:00 INFO HikariDataSource: metastore-monitor - Starting...
25/02/06 06:01:00 INFO HikariDataSource: metastore-monitor - Start completed.
25/02/06 06:01:00 INFO HikariDataSource: metastore-monitor - Shutdown initiated...
25/02/06 06:01:00 INFO HikariDataSource: metastore-monitor - Shutdown completed.
25/02/06 06:01:00 INFO MetastoreMonitor: Metastore healthcheck successful
(connection duration = 323 milliseconds)
25/02/06 06:01:14 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_5594765219939764582_8dd1f651921340808007707781dbd7a0
25/02/06 06:01:14 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_5594765219939764582_8dd1f651921340808007707781dbd7a0
25/02/06 06:01:15 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_5633899408412262627_a3c33e62b62b405bac47b958dccd4554
25/02/06 06:01:15 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_5633899408412262627_a3c33e62b62b405bac47b958dccd4554
25/02/06 06:01:16 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_7337167144654976902_ef479873a51d419caffb2e0977d76b48
25/02/06 06:01:20 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:01:20 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:01:20 WARN DatabricksS3LoggingUtils$:V3: Error calling HeadBucket: 400.
This is typically used to infer the region of the bucket and is harmless. Details:
Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: i-
00f4ec534ef0e57a4 Response Headers: Map(Transfer-Encoding -> chunked, Server ->
AmazonS3, x-amz-id-2 ->
Rzv1I0jW6U94hxFPtLdd7OfZG6EaZCOJImYrtVukcB0IrYmwXVibMhV2gUhYIjqaVrORseVpinE=,
Connection -> close, x-amz-bucket-region -> eu-west-1, Content-Type ->
application/xml, x-amz-request-id -> HMDCMYR6Y693WB3D, Date -> Thu, 06 Feb 2025
06:01:19 GMT)
25/02/06 06:01:20 WARN DatabricksS3LoggingUtils$:V3: Error calling HeadBucket: 400.
This is typically used to infer the region of the bucket and is harmless. Details:
Request ID: null, Extended Request ID: null, Cloud Provider: AWS, Instance ID: i-
00f4ec534ef0e57a4 Response Headers: Map(Transfer-Encoding -> chunked, Server ->
AmazonS3, x-amz-id-2 ->
/mFNW50ack3rYDrQLM9tf9iPLar1MHl9K66TO2Sya6PZ/wWZb/qiZCKpS5kysAH2GQgZ6U6B2fg=,
Connection -> close, x-amz-bucket-region -> eu-west-1, Content-Type ->
application/xml, x-amz-request-id -> HMDBJ7VRK5SYHNKD, Date -> Thu, 06 Feb 2025
06:01:19 GMT)
25/02/06 06:01:20 INFO DeltaLog: Start to prefetch checkpoint for s3://databricks-
workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/
tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/_delta_log
25/02/06 06:01:20 INFO DeltaLog: Start to prefetch checkpoint for s3://databricks-
workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/
tables/21a23d04-c177-49d5-8d77-18da4df94e93/_delta_log
25/02/06 06:01:22 INFO DeltaLog: Loading version 547 starting from checkpoint
version 534.
25/02/06 06:01:22 INFO DeltaLog: Loading version 596 starting from checkpoint
version 400.
25/02/06 06:01:23 INFO SnapshotEdge: [tableId=07222391-4c34-4972-bf73-6f85f7dfe4fb]
Created snapshot SnapshotEdge(path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-
5e2c2c9a7a9e/_delta_log, version=547, metadata=Metadata(22d22904-1726-434e-b13a-
2d5519186a2d,null,null,Format(parquet,Map()),{"type":"struct","fields":
[{"name":"BILL_NUM","type":"string","nullable":true,"metadata":{}},
{"name":"BILL_ITEM","type":"string","nullable":true,"metadata":{}},
{"name":"CALDAY","type":"date","nullable":true,"metadata":{}},
{"name":"RECORDMODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZCHFLAG","type":"string","nullable":true,"metadata":{}},
{"name":"SOLD_TO","type":"string","nullable":true,"metadata":{}},
{"name":"GN_CUSTOM","type":"string","nullable":true,"metadata":{}},
{"name":"SHIP_TO","type":"string","nullable":true,"metadata":{}},
{"name":"BILLTOPRTY","type":"string","nullable":true,"metadata":{}},
{"name":"PAYER","type":"string","nullable":true,"metadata":{}},
{"name":"BP_GRP","type":"string","nullable":true,"metadata":{}},
{"name":"COMP_CODE","type":"string","nullable":true,"metadata":{}},
{"name":"CP_CATEG","type":"string","nullable":true,"metadata":{}},
{"name":"DISTR_CHAN","type":"string","nullable":true,"metadata":{}},
{"name":"DIVISION","type":"string","nullable":true,"metadata":{}},
{"name":"SALESEMPLY","type":"string","nullable":true,"metadata":{}},
{"name":"SALESORG","type":"string","nullable":true,"metadata":{}},
{"name":"SALES_GRP","type":"string","nullable":true,"metadata":{}},
{"name":"SALES_OFF","type":"string","nullable":true,"metadata":{}},
{"name":"MATERIAL","type":"string","nullable":true,"metadata":{}},
{"name":"PRODUCT","type":"string","nullable":true,"metadata":{}},
{"name":"CRM_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"PROD_CATEG","type":"string","nullable":true,"metadata":{}},
{"name":"GN_CAT_SAL","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"PLANT","type":"string","nullable":true,"metadata":{}},
{"name":"GN_LOCATIO","type":"string","nullable":true,"metadata":{}},
{"name":"SHIP_POINT","type":"string","nullable":true,"metadata":{}},
{"name":"DEB_CRED","type":"string","nullable":true,"metadata":{}},
{"name":"IMODOCCAT","type":"string","nullable":true,"metadata":{}},
{"name":"DOC_CLASS","type":"string","nullable":true,"metadata":{}},
{"name":"REFER_ITM","type":"string","nullable":true,"metadata":{}},
{"name":"REFER_DOC","type":"string","nullable":true,"metadata":{}},
{"name":"ITEM_DEL","type":"string","nullable":true,"metadata":{}},
{"name":"GN_R3_SSY","type":"string","nullable":true,"metadata":{}},
{"name":"CALMONTH","type":"string","nullable":true,"metadata":{}},
{"name":"CALQUARTER","type":"string","nullable":true,"metadata":{}},
{"name":"CALYEAR","type":"string","nullable":true,"metadata":{}},
{"name":"FISCVARNT","type":"string","nullable":true,"metadata":{}},
{"name":"FISCPER","type":"string","nullable":true,"metadata":{}},
{"name":"FISCYEAR","type":"string","nullable":true,"metadata":{}},
{"name":"UPD_DATE","type":"date","nullable":true,"metadata":{}},
{"name":"STAT_CURR","type":"string","nullable":true,"metadata":{}},
{"name":"DOC_CURRCY","type":"string","nullable":true,"metadata":{}},
{"name":"BASE_UOM","type":"string","nullable":true,"metadata":{}},
{"name":"BIL_I_CNT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NET_VAL_S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"NETVAL_INV","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"QUANT_B","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"COST_VAL_S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"COST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"GR_WT_KG","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NT_WT_KG","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"VOLUME_CDM","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"SUBTOT_1S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_2S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_3S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_4S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_5S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_6S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"TAX_AMOUNT","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"BILL_DATE","type":"date","nullable":true,"metadata":{}},
{"name":"BILL_TYPE","type":"string","nullable":true,"metadata":{}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"pre_bronze_dlt_delete_flag","type":"string","nullable":true,"metadata":
{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}},{"name":"HEADER_DEL","type":"string","nullable":true,"metadata":{}},
{"name":"DLT_H_CNT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"BIL_H_CNT","type":"decimal(17,3)","nullable":true,"metadata":
{}}]},List(),Map(),Some(1706756224957)), logSegment=LogSegment(s3://databricks-
workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/
tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log,547,WrappedArray(S3AFileStatus{path=s3://databricks-workspace-stack-
f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-
4417-a35d-5e2c2c9a7a9e/_delta_log/00000000000000000535.json; isDirectory=false;
length=556257; replication=1; blocksize=67108864; modification_time=1737862289000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=1106a5e62613c250cdbb91bd4d5a9e1e-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000536.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1737949272000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e64813613a8dd476bcafb19f672b6189-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000537.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738035828000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=96b37fed901924c348593a93d43f06c6-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000538.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738121935000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e9b912c18417b98fefda6410dd7b75eb-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000539.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738208522000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9c2086490a9ed71ce368afb650b481af-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000540.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738296182000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=2a3d743a0b3468031e8e71ed74a9ea75-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000541.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738381825000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b0e492ae121ef4594b79e1bc9bea52ae-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000542.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738468264000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6681e20a460fb36aacfa572f406079fb-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000543.json; isDirectory=false; length=556258;
replication=1; blocksize=67108864; modification_time=1738554624000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=ab4628320a72bfca6283ec18a88a2a0a-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000544.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738641268000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=997e860d525e2d8d51662c62f869e046-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000545.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738727725000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=11e15c116dd1b88a68f923fcc2390f1b-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000546.json; isDirectory=false; length=556257;
replication=1; blocksize=67108864; modification_time=1738804748000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=3c4e238a70d7f7826b0a0eabba416e50-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/
_delta_log/00000000000000000547.json; isDirectory=false; length=556258;
replication=1; blocksize=67108864; modification_time=1738818507000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=8c0fc713ee0f6b6709f3a9aa929b2b79-1
versionId=null),UninitializedV1OrV2ParquetCheckpointProvider(534,S3AFileStatus{path
=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-
1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/_delta_log/
00000000000000000534.checkpoint.parquet; isDirectory=false; length=215195;
replication=1; blocksize=67108864; modification_time=1737777121000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=74799cb3d2d5377c84326d847e4a4efa versionId=null,s3://databricks-workspace-
stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/428f15be-
7119-4417-a35d-5e2c2c9a7a9e/
_delta_log,Some(LastCheckpointInfo(534,1602,None,Some(215195),Some(200),Some(Struct
Type(StructField(txn,StructType(StructField(appId,StringType,true),StructField(vers
ion,LongType,true),StructField(lastUpdated,LongType,true)),true),StructField(add,St
ructType(StructField(path,StringType,true),StructField(partitionValues,MapType(Stri
ngType,StringType,true),true),StructField(size,LongType,true),StructField(modificat
ionTime,LongType,true),StructField(dataChange,BooleanType,true),StructField(tags,Ma
pType(StringType,StringType,true),true),StructField(deletionVector,StructType(Struc
tField(storageType,StringType,true),StructField(pathOrInlineDv,StringType,true),Str
uctField(offset,IntegerType,true),StructField(sizeInBytes,IntegerType,true),StructF
ield(cardinality,LongType,true),StructField(maxRowIndex,LongType,true)),true),Struc
tField(baseRowId,LongType,true),StructField(defaultRowCommitVersion,LongType,true),
StructField(clusteringProvider,StringType,true),StructField(stats,StringType,true),
StructField(stats_parsed,StructType(StructField(numRecords,LongType,true),StructFie
ld(minValues,StructType(StructField(BILL_NUM,StringType,true),StructField(BILL_ITEM
,StringType,true),StructField(CALDAY,DateType,true),StructField(RECORDMODE,StringTy
pe,true),StructField(/BIC/
ZCHFLAG,StringType,true),StructField(SOLD_TO,StringType,true),StructField(GN_CUSTOM
,StringType,true),StructField(SHIP_TO,StringType,true),StructField(BILLTOPRTY,Strin
gType,true),StructField(PAYER,StringType,true),StructField(BP_GRP,StringType,true),
StructField(COMP_CODE,StringType,true),StructField(CP_CATEG,StringType,true),Struct
Field(DISTR_CHAN,StringType,true),StructField(DIVISION,StringType,true),StructField
(SALESEMPLY,StringType,true),StructField(SALESORG,StringType,true),StructField(SALE
S_GRP,StringType,true),StructField(SALES_OFF,StringType,true),StructField(MATERIAL,
StringType,true),StructField(PRODUCT,StringType,true),StructField(CRM_PROD,StringTy
pe,true),StructField(PROD_CATEG,StringType,true),StructField(GN_CAT_SAL,StringType,
true),StructField(MATL_GROUP,StringType,true),StructField(PLANT,StringType,true),St
ructField(GN_LOCATIO,StringType,true),StructField(SHIP_POINT,StringType,true),Struc
tField(DEB_CRED,StringType,true),StructField(IMODOCCAT,StringType,true),StructField
(DOC_CLASS,StringType,true),StructField(REFER_ITM,StringType,true)),true),StructFie
ld(maxValues,StructType(StructField(BILL_NUM,StringType,true),StructField(BILL_ITEM
,StringType,true),StructField(CALDAY,DateType,true),StructField(RECORDMODE,StringTy
pe,true),StructField(/BIC/
ZCHFLAG,StringType,true),StructField(SOLD_TO,StringType,true),StructField(GN_CUSTOM
,StringType,true),StructField(SHIP_TO,StringType,true),StructField(BILLTOPRTY,Strin
gType,true),StructField(PAYER,StringType,true),StructField(BP_GRP,StringType,true),
StructField(COMP_CODE,StringType,true),StructField(CP_CATEG,StringType,true),Struct
Field(DISTR_CHAN,StringType,true),StructField(DIVISION,StringType,true),StructField
(SALESEMPLY,StringType,true),StructField(SALESORG,StringType,true),StructField(SALE
S_GRP,StringType,true),StructField(SALES_OFF,StringType,true),StructField(MATERIAL,
StringType,true),StructField(PRODUCT,StringType,true),StructField(CRM_PROD,StringTy
pe,true),StructField(PROD_CATEG,StringType,true),StructField(GN_CAT_SAL,StringType,
true),StructField(MATL_GROUP,StringType,true),StructField(PLANT,StringType,true),St
ructField(GN_LOCATIO,StringType,true),StructField(SHIP_POINT,StringType,true),Struc
tField(DEB_CRED,StringType,true),StructField(IMODOCCAT,StringType,true),StructField
(DOC_CLASS,StringType,true),StructField(REFER_ITM,StringType,true)),true),StructFie
ld(nullCount,StructType(StructField(BILL_NUM,LongType,true),StructField(BILL_ITEM,L
ongType,true),StructField(CALDAY,LongType,true),StructField(RECORDMODE,LongType,tru
e),StructField(/BIC/
ZCHFLAG,LongType,true),StructField(SOLD_TO,LongType,true),StructField(GN_CUSTOM,Lon
gType,true),StructField(SHIP_TO,LongType,true),StructField(BILLTOPRTY,LongType,true
),StructField(PAYER,LongType,true),StructField(BP_GRP,LongType,true),StructField(CO
MP_CODE,LongType,true),StructField(CP_CATEG,LongType,true),StructField(DISTR_CHAN,L
ongType,true),StructField(DIVISION,LongType,true),StructField(SALESEMPLY,LongType,t
rue),StructField(SALESORG,LongType,true),StructField(SALES_GRP,LongType,true),Struc
tField(SALES_OFF,LongType,true),StructField(MATERIAL,LongType,true),StructField(PRO
DUCT,LongType,true),StructField(CRM_PROD,LongType,true),StructField(PROD_CATEG,Long
Type,true),StructField(GN_CAT_SAL,LongType,true),StructField(MATL_GROUP,LongType,tr
ue),StructField(PLANT,LongType,true),StructField(GN_LOCATIO,LongType,true),StructFi
eld(SHIP_POINT,LongType,true),StructField(DEB_CRED,LongType,true),StructField(IMODO
CCAT,LongType,true),StructField(DOC_CLASS,LongType,true),StructField(REFER_ITM,Long
Type,true)),true)),true)),true),StructField(remove,StructType(StructField(path,Stri
ngType,true),StructField(deletionTimestamp,LongType,true),StructField(dataChange,Bo
oleanType,true),StructField(extendedFileMetadata,BooleanType,true),StructField(part
itionValues,MapType(StringType,StringType,true),true),StructField(size,LongType,tru
e),StructField(deletionVector,StructType(StructField(storageType,StringType,true),S
tructField(pathOrInlineDv,StringType,true),StructField(offset,IntegerType,true),Str
uctField(sizeInBytes,IntegerType,true),StructField(cardinality,LongType,true),Struc
tField(maxRowIndex,LongType,true)),true),StructField(baseRowId,LongType,true),Struc
tField(defaultRowCommitVersion,LongType,true)),true),StructField(metaData,StructTyp
e(StructField(id,StringType,true),StructField(name,StringType,true),StructField(des
cription,StringType,true),StructField(format,StructType(StructField(provider,String
Type,true),StructField(options,MapType(StringType,StringType,true),true)),true),Str
uctField(schemaString,StringType,true),StructField(partitionColumns,ArrayType(Strin
gType,true),true),StructField(configuration,MapType(StringType,StringType,true),tru
e),StructField(createdTime,LongType,true)),true),StructField(protocol,StructType(St
ructField(minReaderVersion,IntegerType,true),StructField(minWriterVersion,IntegerTy
pe,true),StructField(readerFeatures,ArrayType(StringType,true),true),StructField(wr
iterFeatures,ArrayType(StringType,true),true)),true),StructField(domainMetadata,Str
uctType(StructField(domain,StringType,true),StructField(configuration,StringType,tr
ue),StructField(removed,BooleanType,true)),true))),None,None,None,Some(27554c6b9001
89138ba616f5ffdccdd7)))),1738818507000),
checksumOpt=Some(VersionChecksum(Some(d77dade2-75df-4221-973d-
9dd5e19db848),10147868217,200,None,None,1,1,Some(List()),Some(List()),Metadata(22d2
2904-1726-434e-b13a-2d5519186a2d,null,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"BILL_NUM","type":"string","nullable":true,"metadata":{}},
{"name":"BILL_ITEM","type":"string","nullable":true,"metadata":{}},
{"name":"CALDAY","type":"date","nullable":true,"metadata":{}},
{"name":"RECORDMODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZCHFLAG","type":"string","nullable":true,"metadata":{}},
{"name":"SOLD_TO","type":"string","nullable":true,"metadata":{}},
{"name":"GN_CUSTOM","type":"string","nullable":true,"metadata":{}},
{"name":"SHIP_TO","type":"string","nullable":
true,"metadata":{}},
{"name":"BILLTOPRTY","type":"string","nullable":true,"metadata":{}},
{"name":"PAYER","type":"string","nullable":true,"metadata":{}},
{"name":"BP_GRP","type":"string","nullable":true,"metadata":{}},
{"name":"COMP_CODE","type":"string","nullable":true,"metadata":{}},
{"name":"CP_CATEG","type":"string","nullable":true,"metadata":{}},
{"name":"DISTR_CHAN","type":"string","nullable":true,"metadata":{}},
{"name":"DIVISION","type":"string","nullable":true,"metadata":{}},
{"name":"SALESEMPLY","type":"string","nullable":true,"metadata":{}},
{"name":"SALESORG","type":"string","nullable":true,"metadata":{}},
{"name":"SALES_GRP","type":"string","nullable":true,"metadata":{}},
{"name":"SALES_OFF","type":"string","nullable":true,"metadata":{}},
{"name":"MATERIAL","type":"string","nullable":true,"metadata":{}},
{"name":"PRODUCT","type":"string","nullable":true,"metadata":{}},
{"name":"CRM_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"PROD_CATEG","type":"string","nullable":true,"metadata":{}},
{"name":"GN_CAT_SAL","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"PLANT","type":"string","nullable":true,"metadata":{}},
{"name":"GN_LOCATIO","type":"string","nullable":true,"metadata":{}},
{"name":"SHIP_POINT","type":"string","nullable":true,"metadata":{}},
{"name":"DEB_CRED","type":"string","nullable":true,"metadata":{}},
{"name":"IMODOCCAT","type":"string","nullable":true,"metadata":{}},
{"name":"DOC_CLASS","type":"string","nullable":true,"metadata":{}},
{"name":"REFER_ITM","type":"string","nullable":true,"metadata":{}},
{"name":"REFER_DOC","type":"string","nullable":true,"metadata":{}},
{"name":"ITEM_DEL","type":"string","nullable":true,"metadata":{}},
{"name":"GN_R3_SSY","type":"string","nullable":true,"metadata":{}},
{"name":"CALMONTH","type":"string","nullable":true,"metadata":{}},
{"name":"CALQUARTER","type":"string","nullable":true,"metadata":{}},
{"name":"CALYEAR","type":"string","nullable":true,"metadata":{}},
{"name":"FISCVARNT","type":"string","nullable":true,"metadata":{}},
{"name":"FISCPER","type":"string","nullable":true,"metadata":{}},
{"name":"FISCYEAR","type":"string","nullable":true,"metadata":{}},
{"name":"UPD_DATE","type":"date","nullable":true,"metadata":{}},
{"name":"STAT_CURR","type":"string","nullable":true,"metadata":{}},
{"name":"DOC_CURRCY","type":"string","nullable":true,"metadata":{}},
{"name":"BASE_UOM","type":"string","nullable":true,"metadata":{}},
{"name":"BIL_I_CNT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NET_VAL_S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"NETVAL_INV","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"QUANT_B","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"COST_VAL_S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"COST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"GR_WT_KG","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NT_WT_KG","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"VOLUME_CDM","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"SUBTOT_1S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_2S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_3S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_4S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_5S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"SUBTOT_6S","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"TAX_AMOUNT","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"BILL_DATE","type":"date","nullable":true,"metadata":{}},
{"name":"BILL_TYPE","type":"string","nullable":true,"metadata":{}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"pre_bronze_dlt_delete_flag","type":"string","nullable":true,"metadata":
{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}},{"name":"HEADER_DEL","type":"string","nullable":true,"metadata":{}},
{"name":"DLT_H_CNT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"BIL_H_CNT","type":"decimal(17,3)","nullable":true,"metadata":
{}}]},List(),Map(),Some(1706756224957)),Protocol(1,2),Some(FileSizeHistogram(Vector
(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304,
8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736,
41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296,
109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336,
146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024,
251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536,
369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048,
486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560,
603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960,
1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008,
1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560,
2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592,
17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@9b62224,
[J@5c5166a0)),None,None)))
25/02/06 06:01:23 INFO SnapshotEdge: [tableId=265a2335-6dfc-4d56-a19f-031bf8b55942]
Created snapshot SnapshotEdge(path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log, version=596, metadata=Metadata(73a092cd-1660-4df6-83e3-
4db6d538e339,null,null,Format(parquet,Map()),{"type":"struct","fields":
[{"name":"MATERIAL","type":"string","nullable":true,"metadata":{}},
{"name":"OBJVERS","type":"string","nullable":true,"metadata":{}},
{"name":"CHANGED","type":"string","nullable":true,"metadata":{}},
{"name":"AF_COLOR","type":"string","nullable":true,"metadata":{}},
{"name":"AF_FCOCO","type":"string","nullable":true,"metadata":{}},
{"name":"AF_GENDER","type":"string","nullable":true,"metadata":{}},
{"name":"AF_GRID","type":"string","nullable":true,"metadata":{}},
{"name":"AF_STYLE","type":"string","nullable":true,"metadata":{}},
{"name":"APO_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"BASE_UOM","type":"string","nullable":true,"metadata":{}},
{"name":"BASIC_MATL","type":"string","nullable":true,"metadata":{}},
{"name":"BBP_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"COMPETITOR","type":"string","nullable":true,"metadata":{}},
{"name":"CONT_UNIT","type":"string","nullable":true,"metadata":{}},
{"name":"CREATEDON","type":"date","nullable":true,"metadata":{}},
{"name":"CRM_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"DIVISION","type":"string","nullable":true,"metadata":{}},
{"name":"EANUPC","type":"string","nullable":true,"metadata":{}},
{"name":"EXTMATLGRP","type":"string","nullable":true,"metadata":{}},
{"name":"GROSS_CONT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"GROSS_WT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"HC_AGENT1","type":"string","nullable":true,"metadata":{}},
{"name":"HC_AGENT2","type":"string","nullable":true,"metadata":{}},
{"name":"HC_AGENT3","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ANESIND","type":"string","nullable":true,"metadata":{}},
{"name":"HC_APPRTYP","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ATCCODE","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ATCMTYP","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND1","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND2","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND3","type":"string","nullable":true,"metadata":{}},
{"name":"HC_HAZMIND","type":"string","nullable":true,"metadata":{}},
{"name":"HC_IMPMIND","type":"string","nullable":true,"metadata":{}},
{"name":"HEIGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"IND_SECTOR","type":"string","nullable":true,"metadata":{}},
{"name":"LENGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"LOC_CURRCY","type":"string","nullable":true,"metadata":{}},
{"name":"LOGSYS","type":"string","nullable":true,"metadata":{}},
{"name":"MANUFACTOR","type":"string","nullable":true,"metadata":{}},
{"name":"MANU_MATNR","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_CAT","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_TYPE","type":"string","nullable":true,"metadata":{}},
{"name":"MSA_USAGE","type":"string","nullable":true,"metadata":{}},
{"name":"NET_CONT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NET_WEIGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"PO_UNIT","type":"string","nullable":true,"metadata":{}},
{"name":"PROD_HIER","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH1","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH2","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH3","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH4","type":"string","nullable":true,"metadata":{}},
{"name":"RTPLCST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"RT_COLOR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_CONFMAT","type":"string","nullable":true,"metadata":{}},
{"name":"RT_FASHGRD","type":"string","nullable":true,"metadata":{}},
{"name":"RT_MDRELST","type":"string","nullable":true,"metadata":{}},
{"name":"RT_PRBAND","type":"string","nullable":true,"metadata":{}},
{"name":"RT_PRRULE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEAROLL","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEASON","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEASYR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEAYR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SIZE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SUPS","type":"string","nullable":true,"metadata":{}},
{"name":"SIZE_DIM","type":"string","nullable":true,"metadata":{}},
{"name":"STD_DESCR","type":"string","nullable":true,"metadata":{}},
{"name":"UCCERTIFTY","type":"string","nullable":true,"metadata":{}},
{"name":"UCCONSTCLA","type":"string","nullable":true,"metadata":{}},
{"name":"UCFUNCCLAS","type":"string","nullable":true,"metadata":{}},
{"name":"UNIT_DIM","type":"string","nullable":true,"metadata":{}},
{"name":"UNIT_OF_WT","type":"string","nullable":true,"metadata":{}},
{"name":"VENDOR","type":"string","nullable":true,"metadata":{}},
{"name":"VOLUME","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"VOLUMEUNIT","type":"string","nullable":true,"metadata":{}},
{"name":"WIDTH","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"DF_DGNR","type":"string","nullable":true,"metadata":{}},
{"name":"DF_PROFILE","type":"string","nullable":true,"metadata":{}},
{"name":"RF_SIZE2","type":"string","nullable":true,"metadata":{}},
{"name":"RF_BNDID","type":"string","nullable":true,"metadata":{}},
{"name":"RF_FRECHAR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_COLRNGE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SIZRNGE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH2DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH1DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH3DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZSURFPLAT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZFIT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZCUT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZPATTERN","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZORGYEAR","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZORGSEASN","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZGMGCCODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZWARRANTY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZVENART","type":"string","nullable":true,"metadata":{}},
{"name":"UPD_DATE","type":"date","nullable":true,"metadata":{}},{"name":"/BIC/
ZCHFLAG","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBISMT","type":"string","nullable":true,"metadata":{}},
{"name":"RT_VENDP","type":"string","nullable":true,"metadata":{}},
{"name":"PUR_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"VALIDFROM","type":"string","nullable":true,"metadata":{}},
{"name":"VALIDTO","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZPUR_CST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"RT_TAXCODE","type":"string","nullable":true,"metadata":{}},
{"name":"VATTAXTYPE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZDATEFRM","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZDATETO","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMIN_ORD","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"UNIT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMHDHB","type":"decimal(17,3)","nullable":true,"metadata":{}},{"name":"/BIC/
ZBEHVOT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBEHVO","type":"string","nullable":true,"metadata":{}},
{"name":"COUNTRY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBOM","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZHS_CODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZVTEXT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZRSP_CST","type":"decimal(17,2)","nullable":true,"metadata":{}},{"name":"/BIC/
ZEXS_VAL","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"CURRENCY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZNUMTP","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMSTAE","type":"string","nullable":true,"metadata":{}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"pre_bronze_dlt_delete_flag","type":"string","nullable":true,"metadata":
{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}},{"name":"LANGU","type":"string","nullable":true,"metadata":{}},
{"name":"TXTMD","type":"string","nullable":true,"metadata":
{}}]},List(),Map(),Some(1706756197369)), logSegment=LogSegment(s3://databricks-
workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/
tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log,596,WrappedArray(S3AFileStatus{path=s3://databricks-workspace-stack-
f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-
49d5-8d77-18da4df94e93/_delta_log/
00000000000000000401.00000000000000000450.compacted.json; isDirectory=false;
length=110235; replication=1; blocksize=67108864; modification_time=1731041457000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=07aef29522fe015bc7c2ca7b94ea12be versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c
177-49d5-8d77-18da4df94e93/_delta_log/
00000000000000000451.00000000000000000500.compacted.json; isDirectory=false;
length=115357; replication=1; blocksize=67108864; modification_time=1733715616000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=dbe588a9ddb00f06d0011dee6a5c26b9 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000501.00000000000000000550.compacted.json;
isDirectory=false; length=119256; replication=1; blocksize=67108864;
modification_time=1736267604000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=4ac8476b7728b34df9a9d69863730439 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000551.json; isDirectory=false; length=577; replication=1;
blocksize=67108864; modification_time=1736267605000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=586056ee03b969534c6ada06adb40d20
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000552.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1736307941000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9a959bfa4e43820cecec8f02451fc6c2-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000553.json; isDirectory=false; length=619; replication=1;
blocksize=67108864; modification_time=1736355808000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=3be075acfb1826cd0f44e64a0be4900c
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000554.json; isDirectory=false; length=576;
replication=1; blocksize=67108864; modification_time=1736355810000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=7c52843845502cd9fa7b8dedb3ef50b5 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000555.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1736394670000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e473c00e35c807178d5a2a80f5d868e7-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000556.json; isDirectory=false; length=619; replication=1;
blocksize=67108864; modification_time=1736443988000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=9c49e8987a5afcfd30e5e97b6ec01de6
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000557.json; isDirectory=false; length=576;
replication=1; blocksize=67108864; modification_time=1736443991000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=01c9ad62c8f31d71dd47b6f617671895 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000558.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1736481309000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=57d3ab03e80f14aa3068873779bb8575-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000559.json; isDirectory=false; length=618; replication=1;
blocksize=67108864; modification_time=1736495934000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=afe81cea3ec8f9334343966b690be7b3
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000560.json; isDirectory=false; length=575;
replication=1; blocksize=67108864; modification_time=1736495937000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=7c972736f021917b4116e5b790d1e9ba versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000561.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1736567029000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c4fec26d355ff3e23cb8d0b6d199aea7-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000562.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1736653047000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=fe498180952f18e69d8af22f8ab3ddf3-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000563.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1736739314000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=1179894a4681c33a501232f53b2df05d-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000564.json; isDirectory=false; length=21483;
replication=1; blocksize=67108864; modification_time=1736826357000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=0fea08a3172c90214945d56db2fb0eed-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000565.json; isDirectory=false; length=619; replication=1;
blocksize=67108864; modification_time=1736846035000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=0bb12c242e6247530d836e7d54a61185
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000566.json; isDirectory=false; length=576;
replication=1; blocksize=67108864; modification_time=1736846037000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=a94ea542bb115368c486878ca625f7c8 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000567.json; isDirectory=false; length=21480;
replication=1; blocksize=67108864; modification_time=1736913749000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f0274e7d458f33176ad6d93c47b2102a-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000568.json; isDirectory=false; length=21481;
replication=1; blocksize=67108864; modification_time=1737000017000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=ed9a9fd8bcf52a62dc61b27484f07d9a-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-1
8da4df94e93/_delta_log/00000000000000000569.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737084876000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=1d6dc943b2dccd8b4d63688e1f62efd6-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000570.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737171656000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=d15fdef1d5e060b2108b8bc6c2ffb656-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000571.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737256945000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e35823fe94bf10f438321d3749d2e708-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000572.json; isDirectory=false; length=619; replication=1;
blocksize=67108864; modification_time=1737281770000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=341e0236e11a56750a7cd87c8fe46550
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000573.json; isDirectory=false; length=576;
replication=1; blocksize=67108864; modification_time=1737281772000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=21c2326b5abed729015b91dc1f45b07f versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000574.json; isDirectory=false; length=21481;
replication=1; blocksize=67108864; modification_time=1737344029000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=7179460c6852ee3584066e881f0b3888-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000575.json; isDirectory=false; length=21481;
replication=1; blocksize=67108864; modification_time=1737430645000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=842fe91806070131efebc0b8e0f5efe2-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000576.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1737517356000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=bfdb6264721a5b74ccdde2f5b6b6c45b-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000577.json; isDirectory=false; length=618; replication=1;
blocksize=67108864; modification_time=1737545520000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=1a7f937cfb84da7f518ed8e41f94152c
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000578.json; isDirectory=false; length=575;
replication=1; blocksize=67108864; modification_time=1737545522000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e6837dd7441e56fa9d27c41c986443b0 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000579.json; isDirectory=false; length=21480;
replication=1; blocksize=67108864; modification_time=1737604145000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=43c2164eee1694eaa4bb63da108097e1-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000580.json; isDirectory=false; length=619; replication=1;
blocksize=67108864; modification_time=1737635042000; access_time=0; owner=; group=;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE eTag=5bf7839745d8bf0266aebaa4d3cde032
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-
18da4df94e93/_delta_log/00000000000000000581.json; isDirectory=false; length=576;
replication=1; blocksize=67108864; modification_time=1737635045000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=163ac3c1e4b532efeeac1025fa996354 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000582.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737690329000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6d71e5760a43e6dee987b5b110f895c7-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000583.json; isDirectory=false; length=21481;
replication=1; blocksize=67108864; modification_time=1737777115000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=76f7f61398be9fde8884bcaa24baa426-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000584.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737862297000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cf58961648b4c9d432930d1c8784605c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000585.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1737949282000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=7657f5a2508352d91d50abdbf239012e-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000586.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1738035857000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9ad80070fd6e8ed556eecaa5e1a38b97-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000587.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1738121941000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=0a55b8d4efc524fe0a99c90523c25d51-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000588.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1738208556000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=efcbde18c751e7af3b52dc00cd81153d-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000589.json;
isDirectory=false; length=21477; replication=1; blocksize=67108864;
modification_time=1738296190000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=6b591035396c5309a30a359330102a16-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000590.json; isDirectory=false; length=21481;
replication=1; blocksize=67108864; modification_time=1738381828000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=65005285948152f54ae86c44bed077d9-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000591.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1738468271000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e281c970bd981245f745263e85c6236b-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000592.json; isDirectory=false; length=21483;
replication=1; blocksize=67108864; modification_time=1738554649000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cd90784159951e229696e0e12dc988f4-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000593.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1738641273000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9988d96b15517172ee76e6ab40dfc5d5-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000594.json; isDirectory=false; length=21482;
replication=1; blocksize=67108864; modification_time=1738727729000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6dca160bdbde66f7c8f503b545c5a362-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000595.json; isDirectory=false; length=21477;
replication=1; blocksize=67108864; modification_time=1738804756000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=19b804651e31d487b41eadb0d158ff00-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/
_delta_log/00000000000000000596.json; isDirectory=false; length=21483;
replication=1; blocksize=67108864; modification_time=1738818521000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=78b53fc97be42bc64e619ee5c266edd8-1
versionId=null),UninitializedV1OrV2ParquetCheckpointProvider(400,S3AFileStatus{path
=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-
1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/_delta_log/
00000000000000000400.checkpoint.parquet; isDirectory=false; length=61194;
replication=1; blocksize=67108864; modification_time=1728794849000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=78ac96df69cf2ec9fea0455034accf73 versionId=null,s3://databricks-workspace-
stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/21a23d04-
c177-49d5-8d77-18da4df94e93/
_delta_log,Some(LastCheckpointInfo(400,98,None,Some(61194),Some(8),Some(StructType(
StructField(txn,StructType(StructField(appId,StringType,true),StructField(version,L
ongType,true),StructField(lastUpdated,LongType,true)),true),StructField(add,StructT
ype(StructField(path,StringType,true),StructField(partitionValues,MapType(StringTyp
e,StringType,true),true),StructField(size,LongType,true),StructField(modificationTi
me,LongType,true),StructField(dataChange,BooleanType,true),StructField(tags,MapType
(StringType,StringType,true),true),StructField(deletionVector,StructType(StructFiel
d(storageType,StringType,true),StructField(pathOrInlineDv,StringType,true),StructFi
eld(offset,IntegerType,true),StructField(sizeInBytes,IntegerType,true),StructField(
cardinality,LongType,true),StructField(maxRowIndex,LongType,true)),true),StructFiel
d(baseRowId,LongType,true),StructField(defaultRowCommitVersion,LongType,true),Struc
tField(clusteringProvider,StringType,true),StructField(stats,StringType,true),Struc
tField(stats_parsed,StructType(StructField(numRecords,LongType,true),StructField(mi
nValues,StructType(StructField(MATERIAL,StringType,true),StructField(OBJVERS,String
Type,true),StructField(CHANGED,StringType,true),StructField(AF_COLOR,StringType,tru
e),StructField(AF_FCOCO,StringType,true),StructField(AF_GENDER,StringType,true),Str
uctField(AF_GRID,StringType,true),StructField(AF_STYLE,StringType,true),StructField
(APO_PROD,StringType,true),StructField(BASE_UOM,StringType,true),StructField(BASIC_
MATL,StringType,true),StructField(BBP_PROD,StringType,true),StructField(COMPETITOR,
StringType,true),StructField(CONT_UNIT,StringType,true),StructField(CREATEDON,DateT
ype,true),StructField(CRM_PROD,StringType,true),StructField(DIVISION,StringType,tru
e),StructField(EANUPC,StringType,true),StructField(EXTMATLGRP,StringType,true),Stru
ctField(GROSS_CONT,DecimalType(17,3),true),StructField(GROSS_WT,DecimalType(17,3),t
rue),StructField(HC_AGENT1,StringType,true),StructField(HC_AGENT2,StringType,true),
StructField(HC_AGENT3,StringType,true),StructField(HC_ANESIND,StringType,true),Stru
ctField(HC_APPRTYP,StringType,true),StructField(HC_ATCCODE,StringType,true),StructF
ield(HC_ATCMTYP,StringType,true),StructField(HC_CATIND1,StringType,true),StructFiel
d(HC_CATIND2,StringType,true),StructField(HC_CATIND3,StringType,true),StructField(H
C_HAZMIND,StringType,true)),true),StructField(maxValues,StructType(StructField(MATE
RIAL,StringType,true),StructField(OBJVERS,StringType,true),StructField(CHANGED,Stri
ngType,true),StructField(AF_COLOR,StringType,true),StructField(AF_FCOCO,StringType,
true),StructField(AF_GENDER,StringType,true),StructField(AF_GRID,StringType,true),S
tructField(AF_STYLE,StringType,true),StructField(APO_PROD,StringType,true),StructFi
eld(BASE_UOM,StringType,true),StructField(BASIC_MATL,StringType,true),StructField(B
BP_PROD,StringType,true),StructField(COMPETITOR,StringType,true),StructField(CONT_U
NIT,StringType,true),StructField(CREATEDON,DateType,true),StructField(CRM_PROD,Stri
ngType,true),StructField(DIVISION,StringType,true),StructField(EANUPC,StringType,tr
ue),StructField(EXTMATLGRP,StringType,true),StructField(GROSS_CONT,DecimalType(17,3
),true),StructField(GROSS_WT,DecimalType(17,3),true),StructField(HC_AGENT1,StringTy
pe,true),StructField(HC_AGENT2,StringType,true),StructField(HC_AGENT3,StringType,tr
ue),StructField(HC_ANESIND,StringType,true),StructField(HC_APPRTYP,StringType,true)
,StructField(HC_ATCCODE,StringType,true),StructField(HC_ATCMTYP,StringType,true),St
ructField(HC_CATIND1,StringType,true),StructField(HC_CATIND2,StringType,true),Struc
tField(HC_CATIND3,StringType,true),StructField(HC_HAZMIND,StringType,true)),true),S
tructField(nullCount,StructType(StructField(MATERIAL,LongType,true),StructField(OBJ
VERS,LongType,true),StructField(CHANGED,LongType,true),StructField(AF_COLOR,LongTyp
e,true),StructField(AF_FCOCO,LongType,true),StructField(AF_GENDER,LongType,true),St
ructField(AF_GRID,LongType,true),StructField(AF_STYLE,LongType,true),StructField(AP
O_PROD,LongType,true),StructField(BASE_UOM,LongType,true),StructField(BASIC_MATL,Lo
ngType,true),StructField(BBP_PROD,LongType,true),StructField(COMPETITOR,LongType,tr
ue),StructField(CONT_UNIT,LongType,true),StructField(CREATEDON,LongType,true),Struc
tField(CRM_PROD,LongType,true),StructField(DIVISION,LongType,true),StructField(EANU
PC,LongType,true),StructField(EXTMATLGRP,LongType,true),StructField(GROSS_CONT,Long
Type,true),StructField(GROSS_WT,LongType,true),StructField(HC_AGENT1,LongType,true)
,StructField(HC_AGENT2,LongType,true),StructField(HC_AGENT3,LongType,true),StructFi
eld(HC_ANESIND,LongType,true),StructField(HC_APPRTYP,LongType,true),StructField(HC_
ATCCODE,LongType,true),StructField(HC_ATCMTYP,LongType,true),StructField(HC_CATIND1
,LongType,true),StructField(HC_CATIND2,LongType,true),StructField(HC_CATIND3,LongTy
pe,true),StructField(HC_HAZMIND,LongType,true)),true)),true)),true),StructField(rem
ove,StructType(StructField(path,StringType,true),StructField(deletionTimestamp,Long
Type,true),StructField(dataChange,BooleanType,true),StructField(extendedFileMetadat
a,BooleanType,true),StructField(partitionValues,MapType(StringType,StringType,true)
,true),StructField(size,LongType,true),StructField(deletionVector,StructType(Struct
Field(storageType,StringType,true),StructField(pathOrInlineDv,StringType,true),Stru
ctField(offset,IntegerType,true),StructField(sizeInBytes,IntegerType,true),StructFi
eld(cardinality,LongType,true),StructField(maxRowIndex,LongType,true)),true),Struct
Field(baseRowId,LongType,true),Stru
ctField(defaultRowCommitVersion,LongType,true)),true),StructField(metaData,StructTy
pe(StructField(id,StringType,true),StructField(name,StringType,true),StructField(de
scription,StringType,true),StructField(format,StructType(StructField(provider,Strin
gType,true),StructField(options,MapType(StringType,StringType,true),true)),true),St
ructField(schemaString,StringType,true),StructField(partitionColumns,ArrayType(Stri
ngType,true),true),StructField(configuration,MapType(StringType,StringType,true),tr
ue),StructField(createdTime,LongType,true)),true),StructField(protocol,StructType(S
tructField(minReaderVersion,IntegerType,true),StructField(minWriterVersion,IntegerT
ype,true),StructField(readerFeatures,ArrayType(StringType,true),true),StructField(w
riterFeatures,ArrayType(StringType,true),true)),true),StructField(domainMetadata,St
ructType(StructField(domain,StringType,true),StructField(configuration,StringType,t
rue),StructField(removed,BooleanType,true)),true))),None,None,None,Some(bb0bd2f9a08
da468c5e4f8418b6bd489)))),1738818521000),
checksumOpt=Some(VersionChecksum(Some(21340df6-0ac7-4798-b7ee-
a667c3b91c12),305197938,8,None,None,1,1,Some(List()),Some(List()),Metadata(73a092cd
-1660-4df6-83e3-4db6d538e339,null,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"MATERIAL","type":"string","nullable":true,"metadata":{}},
{"name":"OBJVERS","type":"string","nullable":true,"metadata":{}},
{"name":"CHANGED","type":"string","nullable":true,"metadata":{}},
{"name":"AF_COLOR","type":"string","nullable":true,"metadata":{}},
{"name":"AF_FCOCO","type":"string","nullable":true,"metadata":{}},
{"name":"AF_GENDER","type":"string","nullable":true,"metadata":{}},
{"name":"AF_GRID","type":"string","nullable":true,"metadata":{}},
{"name":"AF_STYLE","type":"string","nullable":true,"metadata":{}},
{"name":"APO_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"BASE_UOM","type":"string","nullable":true,"metadata":{}},
{"name":"BASIC_MATL","type":"string","nullable":true,"metadata":{}},
{"name":"BBP_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"COMPETITOR","type":"string","nullable":true,"metadata":{}},
{"name":"CONT_UNIT","type":"string","nullable":true,"metadata":{}},
{"name":"CREATEDON","type":"date","nullable":true,"metadata":{}},
{"name":"CRM_PROD","type":"string","nullable":true,"metadata":{}},
{"name":"DIVISION","type":"string","nullable":true,"metadata":{}},
{"name":"EANUPC","type":"string","nullable":true,"metadata":{}},
{"name":"EXTMATLGRP","type":"string","nullable":true,"metadata":{}},
{"name":"GROSS_CONT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"GROSS_WT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"HC_AGENT1","type":"string","nullable":true,"metadata":{}},
{"name":"HC_AGENT2","type":"string","nullable":true,"metadata":{}},
{"name":"HC_AGENT3","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ANESIND","type":"string","nullable":true,"metadata":{}},
{"name":"HC_APPRTYP","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ATCCODE","type":"string","nullable":true,"metadata":{}},
{"name":"HC_ATCMTYP","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND1","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND2","type":"string","nullable":true,"metadata":{}},
{"name":"HC_CATIND3","type":"string","nullable":true,"metadata":{}},
{"name":"HC_HAZMIND","type":"string","nullable":true,"metadata":{}},
{"name":"HC_IMPMIND","type":"string","nullable":true,"metadata":{}},
{"name":"HEIGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"IND_SECTOR","type":"string","nullable":true,"metadata":{}},
{"name":"LENGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"LOC_CURRCY","type":"string","nullable":true,"metadata":{}},
{"name":"LOGSYS","type":"string","nullable":true,"metadata":{}},
{"name":"MANUFACTOR","type":"string","nullable":true,"metadata":{}},
{"name":"MANU_MATNR","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_CAT","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"MATL_TYPE","type":"string","nullable":true,"metadata":{}},
{"name":"MSA_USAGE","type":"string","nullable":true,"metadata":{}},
{"name":"NET_CONT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"NET_WEIGHT","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"PO_UNIT","type":"string","nullable":true,"metadata":{}},
{"name":"PROD_HIER","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH1","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH2","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH3","type":"string","nullable":true,"metadata":{}},
{"name":"RPA_WGH4","type":"string","nullable":true,"metadata":{}},
{"name":"RTPLCST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"RT_COLOR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_CONFMAT","type":"string","nullable":true,"metadata":{}},
{"name":"RT_FASHGRD","type":"string","nullable":true,"metadata":{}},
{"name":"RT_MDRELST","type":"string","nullable":true,"metadata":{}},
{"name":"RT_PRBAND","type":"string","nullable":true,"metadata":{}},
{"name":"RT_PRRULE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEAROLL","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEASON","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEASYR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SEAYR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SIZE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SUPS","type":"string","nullable":true,"metadata":{}},
{"name":"SIZE_DIM","type":"string","nullable":true,"metadata":{}},
{"name":"STD_DESCR","type":"string","nullable":true,"metadata":{}},
{"name":"UCCERTIFTY","type":"string","nullable":true,"metadata":{}},
{"name":"UCCONSTCLA","type":"string","nullable":true,"metadata":{}},
{"name":"UCFUNCCLAS","type":"string","nullable":true,"metadata":{}},
{"name":"UNIT_DIM","type":"string","nullable":true,"metadata":{}},
{"name":"UNIT_OF_WT","type":"string","nullable":true,"metadata":{}},
{"name":"VENDOR","type":"string","nullable":true,"metadata":{}},
{"name":"VOLUME","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"VOLUMEUNIT","type":"string","nullable":true,"metadata":{}},
{"name":"WIDTH","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"DF_DGNR","type":"string","nullable":true,"metadata":{}},
{"name":"DF_PROFILE","type":"string","nullable":true,"metadata":{}},
{"name":"RF_SIZE2","type":"string","nullable":true,"metadata":{}},
{"name":"RF_BNDID","type":"string","nullable":true,"metadata":{}},
{"name":"RF_FRECHAR","type":"string","nullable":true,"metadata":{}},
{"name":"RT_COLRNGE","type":"string","nullable":true,"metadata":{}},
{"name":"RT_SIZRNGE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH2DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH1DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZAH3DESC","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZSURFPLAT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZFIT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZCUT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZPATTERN","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZORGYEAR","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZORGSEASN","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZGMGCCODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZWARRANTY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZVENART","type":"string","nullable":true,"metadata":{}},
{"name":"UPD_DATE","type":"date","nullable":true,"metadata":{}},{"name":"/BIC/
ZCHFLAG","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBISMT","type":"string","nullable":true,"metadata":{}},
{"name":"RT_VENDP","type":"string","nullable":true,"metadata":{}},
{"name":"PUR_GROUP","type":"string","nullable":true,"metadata":{}},
{"name":"VALIDFROM","type":"string","nullable":true,"metadata":{}},
{"name":"VALIDTO","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZPUR_CST","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"RT_TAXCODE","type":"string","nullable":true,"metadata":{}},
{"name":"VATTAXTYPE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZDATEFRM","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZDATETO","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMIN_ORD","type":"decimal(17,3)","nullable":true,"metadata":{}},
{"name":"UNIT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMHDHB","type":"decimal(17,3)","nullable":true,"metadata":{}},{"name":"/BIC/
ZBEHVOT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBEHVO","type":"string","nullable":true,"metadata":{}},
{"name":"COUNTRY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZBOM","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZHS_CODE","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZVTEXT","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZRSP_CST","type":"decimal(17,2)","nullable":true,"metadata":{}},{"name":"/BIC/
ZEXS_VAL","type":"decimal(17,2)","nullable":true,"metadata":{}},
{"name":"CURRENCY","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZNUMTP","type":"string","nullable":true,"metadata":{}},{"name":"/BIC/
ZMSTAE","type":"string","nullable":true,"metadata":{}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"pre_bronze_dlt_delete_flag","type":"string","nullable":true,"metadata":
{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}},{"name":"LANGU","type":"string","nullable":true,"metadata":{}},
{"name":"TXTMD","type":"string","nullable":true,"metadata":{}}]},
List(),Map(),Some(1706756197369)),Protocol(1,2),Some(FileSizeHistogram(Vector(0,
8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304,
8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736,
41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296,
109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336,
146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024,
251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536,
369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048,
486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560,
603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960,
1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008,
1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560,
2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592,
17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@15e93d9c,
[J@28392967)),None,Some(List(AddFile(part-00005-b9e223a3-a920-4c81-9aae-
65349ab5630a.c000.snappy.parquet,Map(),41215765,1738818516000,false,
{"numRecords":636535,"minValues":
{"MATERIAL":"000000000000000000","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":{"MATERIAL":" PERFORATED VEG
RO","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","AF_GENDER":"","AF_GRID"
:"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL":"","BBP_PROD":"","COM
PETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"80","EANUPC":"
7350125970089","EXTMATLGRP":"T0611","GR
OSS_CONT":0.000,"GROSS_WT":4752.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","H
C_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_C
ATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":63859,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000005,
MIN_INSERTION_TIME -> 1738818516000005, MAX_INSERTION_TIME -> 1738818516000005,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00004-
1ac905e1-1d11-402e-ae5f-
dc1f1d0ac8f0.c000.snappy.parquet,Map(),41106760,1738818516000,false,
{"numRecords":637146,"minValues":
{"MATERIAL":"000000000000030012","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":
{"MATERIAL":"ZZYV1451BB01","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","
AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL
":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"85","EANUPC":"
7350125970096","EXTMATLGRP":"T0611","GR
OSS_CONT":0.000,"GROSS_WT":1750.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","H
C_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_C
ATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":63645,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000004,
MIN_INSERTION_TIME -> 1738818516000004, MAX_INSERTION_TIME -> 1738818516000004,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00001-
3c364953-6b3c-4043-a19d-
66b330e6acd2.c000.snappy.parquet,Map(),41136603,1738818516000,false,
{"numRecords":637563,"minValues":
{"MATERIAL":"000000000000030001","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":{"MATERIAL":" ROUND CLEAR
SALAD","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","AF_GENDER":"","AF_GR
ID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL":"","BBP_PROD":"","
COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"80","EANUPC":"6294001007364","EXTMATLGRP":"GENERIC
SS","GROSS_CONT":0.000,"GROSS_WT":4752.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3
":"","HC_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"
","HC_CATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":63639,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000001,
MIN_INSERTION_TIME -> 1738818516000001, MAX_INSERTION_TIME -> 1738818516000001,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00002-
ce9391f0-f82a-440e-8740-
654cf6629505.c000.snappy.parquet,Map(),41029515,1738818517000,false,
{"numRecords":636308,"minValues":
{"MATERIAL":"000000000000030000","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":
{"MATERIAL":"ZZZ9253NAT01","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","
AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL
":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"85","EANUPC":"6294001007241","EXTMATLGRP":"T0611","GR
OSS_CONT":0.000,"GROSS_WT":1633.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","H
C_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_C
ATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":63901,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000002,
MIN_INSERTION_TIME -> 1738818516000002, MAX_INSERTION_TIME -> 1738818516000002,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00007-
f2746f3e-1e28-4f8b-9203-
687fe77e38bd.c000.snappy.parquet,Map(),17095737,1738818516000,false,
{"numRecords":260198,"minValues":
{"MATERIAL":"000000000000000011","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":
{"MATERIAL":"ZZWV2701BA097201","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":
"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_
MATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"80","EANUPC":" 850008289057","EXTMATLGRP":"GENERIC
SS","GROSS_CONT":0.000,"GROSS_WT":800.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3"
:"","HC_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":""
,"HC_CATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":26074,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT
":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_ANESIND":0,"HC_APPRTYP":0,"HC_ATC
CODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0,"HC_CATIND3":0,"HC_HAZMIND":0}
},Map(INSERTION_TIME -> 1738818516000007, MIN_INSERTION_TIME -> 1738818516000007,
MAX_INSERTION_TIME -> 1738818516000007, OPTIMIZE_TARGET_SIZE ->
67108864),null,None,None,None), AddFile(part-00003-5aeb3b61-70bc-4fb7-9bd8-
90a3d20ec5ab.c000.snappy.parquet,Map(),41231344,1738818516000,false,
{"numRecords":637618,"minValues":
{"MATERIAL":"000000000000030002","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":
{"MATERIAL":"ZZWU1305LO276001","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":
"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_
MATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"80","EANUPC":"
7350125970027","EXTMATLGRP":"GENERIC
SS","GROSS_CONT":0.000,"GROSS_WT":2020.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3
":"","HC_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"
","HC_CATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":63937,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000003,
MIN_INSERTION_TIME -> 1738818516000003, MAX_INSERTION_TIME -> 1738818516000003,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00006-
2684b608-fed7-4b19-a3de-
8307fce1f34c.c000.snappy.parquet,Map(),41186803,1738818516000,false,
{"numRecords":637174,"minValues":
{"MATERIAL":"000000000000030006","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":
{"MATERIAL":"ZZZ9253NAT","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","AF
_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL":
"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"85","EANUPC":"
7350125970126","EXTMATLGRP":"GENERIC
SS","GROSS_CONT":0.000,"GROSS_WT":1420.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3
":"","HC_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"
","HC_CATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":64003,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000006,
MIN_INSERTION_TIME -> 1738818516000006, MAX_INSERTION_TIME -> 1738818516000006,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00000-
5a1b9bab-e3a9-4c7c-8e7e-
691883bed26b.c000.snappy.parquet,Map(),41195411,1738818516000,false,
{"numRecords":638336,"minValues":
{"MATERIAL":"000000000000030003","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO
":"","AF_GENDER":"","AF_GRID":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"","BASIC_M
ATL":"","BBP_PROD":"","COMPETITOR":"","CONT_UNIT":"","CREATEDON":"2015-04-
05","CRM_PROD":"","DIVISION":"","EANUPC":"","EXTMATLGRP":"","GROSS_CONT":0.000,"GRO
SS_WT":0.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","HC_ANESIND":"","HC_APPRT
YP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_CATIND2":"","HC_CATIND3"
:"","HC_HAZMIND":""},"maxValues":{"MATERIAL":"ZZZALLAN-
P08","OBJVERS":"A","CHANGED":"","AF_COLOR":"","AF_FCOCO":"","AF_GENDER":"","AF_GRID
":"","AF_STYLE":"","APO_PROD":"","BASE_UOM":"YTR","BASIC_MATL":"","BBP_PROD":"","CO
MPETITOR":"","CONT_UNIT":"ST","CREATEDON":"2025-02-
05","CRM_PROD":"","DIVISION":"80","EANUPC":"6294001007401","EXTMATLGRP":"T0611","GR
OSS_CONT":0.000,"GROSS_WT":1225.000,"HC_AGENT1":"","HC_AGENT2":"","HC_AGENT3":"","H
C_ANESIND":"","HC_APPRTYP":"","HC_ATCCODE":"","HC_ATCMTYP":"","HC_CATIND1":"","HC_C
ATIND2":"","HC_CATIND3":"","HC_HAZMIND":""},"nullCount":
{"MATERIAL":0,"OBJVERS":0,"CHANGED":0,"AF_COLOR":0,"AF_FCOCO":0,"AF_GENDER":0,"AF_G
RID":0,"AF_STYLE":0,"APO_PROD":0,"BASE_UOM":0,"BASIC_MATL":0,"BBP_PROD":0,"COMPETIT
OR":0,"CONT_UNIT":0,"CREATEDON":64154,"CRM_PROD":0,"DIVISION":0,"EANUPC":0,"EXTMATL
GRP":0,"GROSS_CONT":0,"GROSS_WT":0,"HC_AGENT1":0,"HC_AGENT2":0,"HC_AGENT3":0,"HC_AN
ESIND":0,"HC_APPRTYP":0,"HC_ATCCODE":0,"HC_ATCMTYP":0,"HC_CATIND1":0,"HC_CATIND2":0
,"HC_CATIND3":0,"HC_HAZMIND":0}},Map(INSERTION_TIME -> 1738818516000000,
MIN_INSERTION_TIME -> 1738818516000000, MAX_INSERTION_TIME -> 1738818516000000,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None))))))
25/02/06 06:01:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:01:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:01:23 INFO DriverCorral: DBFS health check ok
25/02/06 06:01:23 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:01:23 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:01:23 INFO HiveMetaStore: 1: Opening raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
25/02/06 06:01:23 INFO ObjectStore: ObjectStore, initialize called
25/02/06 06:01:24 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:01:24 INFO ObjectStore: Initialized ObjectStore
25/02/06 06:01:24 INFO DriverCorral: Metastore health check ok
25/02/06 06:01:24 INFO DeltaLog: Loading version 132.
25/02/06 06:01:24 INFO SnapshotEdge: [tableId=b7bdbb8a-0265-4045-88d9-6ffa6271d612]
Created snapshot SnapshotEdge(path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-
7df092bb2727/_delta_log, version=132, metadata=Metadata(4e6ba28d-7c39-4c4a-9add-
fe833f23e20f,__materialization_mat_bpp_a155_1,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"MANDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KAPPL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSCHL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VKORG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VTWEG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"PLTYP","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MATNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VRKME","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"DATBI","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"DATAB","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}}]},List(),Map(pipeline_internal.catalogType -> UNITY_CATALOG,
delta.enableChangeDataFeed -> true, pipelines.pipelineId -> 2ba61ec2-c859-4b46-
ab90-5e6b1e15c4f3, pipelines.metastore.tableName ->
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`,
spark.internal.materialized_view.parentTable ->
`prod_mena`.`bronze`.`bpp_a155`),Some(1728100402013)),
logSegment=LogSegment(s3://databricks-workspace-stack-f2c65-metastore-bucket/
7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log,132,WrappedArray(S3AFileStatus{path=s3://databricks-workspace-stack-
f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-
49f1-b8af-7df092bb2727/_delta_log/00000000000000000000.json; isDirectory=false;
length=869; replication=1; blocksize=67108864; modification_time=1728100403000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=259a417c8906699de4d7ca6ef641358b versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000001.00000000000000000050.compacted.json;
isDirectory=false; length=40346; replication=1; blocksize=67108864;
modification_time=1732160535000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=0addc046773ada892e0241296e94f48d versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000051.00000000000000000100.compacted.json;
isDirectory=false; length=39860; replication=1; blocksize=67108864;
modification_time=1736392443000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=59a75936bdda1dd108554d8bb2feb9d8 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000101.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1736479001000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=490a6bd23b29ae3a31c76f52f6aa993c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000102.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1736513098000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9a418c9bbb000f446f16fb0a4cd33c40-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000103.json; isDirectory=false; length=4386;
replication=1; blocksize=67108864; modification_time=1736564976000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cb4677731cf6c8bd06744604497277cf-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000104.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1736651382000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c5eb248700ed2612eb9a5ad0d4bfe05c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000105.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1736737979000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=04c2b07fec5f866c8b0c5d65e98119ad-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000106.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1736824377000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=4150ff9f822cd6d81acc7fe3bd0359f7-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000107.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1736911536000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=8448ea77567f21db180d6417e98bb4ce-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000108.json; isDirectory=false; length=2551;
replication=1; blocksize=67108864; modification_time=1736996387000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=942b17158e6ca57de5d4ed4311a529d1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000109.json; isDirectory=false; length=2457;
replication=1; blocksize=67108864; modification_time=1736996503000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=74118529c0f04294e706935aa28ddd66 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000110.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1736997826000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=480267900aedd775cdc80c7975286ca8-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000111.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737082911000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=59010be9cae296805e9233eef6e07ce4-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000112.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737169499000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f9381e891e6ac0405f48d876f0ed0ee8-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000113.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1737255055000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c7fff5ddfcd20cc09af6409991a0b890-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000114.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737342174000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6e1f3bc2716e5b2781a93a797903e9af-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000115.json; isDirectory=false; length=4389;
replication=1; blocksize=67108864; modification_time=1737428539000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6b949d567a53fc0d59bff7fa568183a3-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000116.json; isDirectory=false; length=4386;
replication=1; blocksize=67108864; modification_time=1737515078000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=fc945e2d90b5642e4abb5b26579f5c2d-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000117.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737601795000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cd57275743c11ca63528425bf17d54fb-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000118.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737688226000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c9a21df30878000e04d1d557f9099c9b-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000119.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1737775011000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f37c9c023c020144514128be11c86f8c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000120.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1737860436000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=77dc018998c3a370edf0cb9e110c8aa0-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000121.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1737947583000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=1a54fe13de3be1def6b7f055561ee07d-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000122.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738033487000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f0ddb17ca038d6198b139d83778084d6-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000123.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738119722000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=51b9f8a836253286ef144a913edbe15c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000124.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1738206149000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f59a14a06fc795da6a56fc32978d8856-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000125.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1738293826000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=16b1c98ac60db2c075c9c5c21a746bfa-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000126.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738379498000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=34b3a4a6337b053ff88cc4e848a141a6-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000127.json; isDirectory=false; length=4388;
replication=1; blocksize=67108864; modification_time=1738466416000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=6c3db961018b9f8ee6cfa37c31f35502-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000128.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738552862000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=0ae3ca1ed8182589a26c3559b0fcea39-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000129.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738639284000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=02b0e8b582e010e187cd3ee7513d3bfd-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000130.json; isDirectory=false; length=4390;
replication=1; blocksize=67108864; modification_time=1738725318000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cb6f8277ca4ddb35cadab78b444643cf-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000131.json; isDirectory=false; length=4386;
replication=1; blocksize=67108864; modification_time=1738802414000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=62a7d3c8b7a1bd8a26ac9dafd3db5c29-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/1f4ff2dd-0c45-49f1-b8af-7df092bb2727/
_delta_log/00000000000000000132.json; isDirectory=false; length=4395;
replication=1; blocksize=67108864; modification_time=1738817026000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=fadf6f9749a7bf6f171a43d39c3214a4-1
versionId=null),com.databricks.sql.transaction.tahoe.EmptyCheckpointProvider$@61a0d
d95,1738817026000), checksumOpt=Some(VersionChecksum(Some(d997e827-f4d0-4785-ba03-
8f7c52503291),616647619,2,None,None,1,1,Some(List(SetTransaction(com.databricks.pip
elines.execution.core.enzyme.provenance,1738817025488,Some(1738817025488)))),Some(L
ist()),Metadata(4e6ba28d-7c39-4c4a-9add-fe833f23e20
f,__materialization_mat_bpp_a155_1,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"MANDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KAPPL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSCHL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VKORG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VTWEG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"PLTYP","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MATNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VRKME","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"DATBI","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"DATAB","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}},
{"name":"bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metadata"
:{}}]},List(),Map(pipeline_internal.catalogType -> UNITY_CATALOG,
delta.enableChangeDataFeed -> true, pipelines.pipelineId -> 2ba61ec2-c859-4b46-
ab90-5e6b1e15c4f3, pipelines.metastore.tableName ->
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`,
spark.internal.materialized_view.parentTable ->
`prod_mena`.`bronze`.`bpp_a155`),Some(1728100402013)),Protocol(1,4),Some(FileSizeHi
stogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576,
2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128,
33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080,
92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728,
138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592,
218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104,
335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616,
452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128,
570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232,
939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280,
1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648,
2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840,
4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472,
274877906944),[J@6f67b73b,[J@5e55376c)),None,Some(List(AddFile(part-00001-8ba33aa3-
de25-4ea5-9cdd-
5d744cf93c44.c000.snappy.parquet,Map(),245076766,1738817013000,false,
{"numRecords":13430778,"minValues":
{"MANDT":"300","KAPPL":"V","KSCHL":"VKP0","VKORG":"1010","VTWEG":"10","PLTYP":"CR",
"MATNR":"000000000022223784","VRKME":"","DATBI":"20160129","DATAB":"02170101","KNUM
H":"0003087680","landing_update_timestamp":"2024-10-
01T14:21:24.061Z","pre_bronze_dlt_update_timestamp":"2025-02-
06T03:54:02.570Z","bronze_dlt_update_timestamp":"2025-02-
06T04:36:35.005Z"},"maxValues":
{"MANDT":"300","KAPPL":"V","KSCHL":"VKP0","VKORG":"8500","VTWEG":"30","PLTYP":"WS",
"MATNR":"ZZWV2701BA0972","VRKME":"ST","DATBI":"99991231","DATAB":"20240814","KNUMH"
:"0061486744","landing_update_timestamp":"2024-10-
01T14:21:24.061Z","pre_bronze_dlt_update_timestamp":"2025-02-
06T03:54:02.570Z","bronze_dlt_update_timestamp":"2025-02-
06T04:36:35.005Z"},"nullCount":
{"MANDT":0,"KAPPL":0,"KSCHL":0,"VKORG":0,"VTWEG":0,"PLTYP":0,"MATNR":0,"VRKME":0,"D
ATBI":0,"DATAB":0,"KNUMH":0,"landing_update_timestamp":0,"pre_bronze_dlt_update_tim
estamp":0,"bronze_dlt_update_timestamp":0}},Map(INSERTION_TIME -> 1738817013000001,
MIN_INSERTION_TIME -> 1738817013000001, MAX_INSERTION_TIME -> 1738817013000001,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None), AddFile(part-00000-
9d9398e4-556e-4e78-82b9-
f45edae3cdc0.c000.snappy.parquet,Map(),371570853,1738817013000,false,
{"numRecords":20355778,"minValues":
{"MANDT":"300","KAPPL":"V","KSCHL":"VKP0","VKORG":"1010","VTWEG":"10","PLTYP":"CR",
"MATNR":" NKCZ4152-
101","VRKME":"","DATBI":"20160129","DATAB":"20160129","KNUMH":"0003087679","landing
_update_timestamp":"2024-10-
01T14:21:24.061Z","pre_bronze_dlt_update_timestamp":"2025-02-
06T03:54:02.570Z","bronze_dlt_update_timestamp":"2025-02-
06T04:36:35.005Z"},"maxValues":
{"MANDT":"300","KAPPL":"V","KSCHL":"VKP2","VKORG":"8500","VTWEG":"30","PLTYP":"WS",
"MATNR":"ZZZALLAN-
P08","VRKME":"ST","DATBI":"99991231","DATAB":"20240814","KNUMH":"0061486746","landi
ng_update_timestamp":"2024-10-
01T14:21:24.061Z","pre_bronze_dlt_update_timestamp":"2025-02-
06T03:54:02.570Z","bronze_dlt_update_timestamp":"2025-02-
06T04:36:35.005Z"},"nullCount":
{"MANDT":0,"KAPPL":0,"KSCHL":0,"VKORG":0,"VTWEG":0,"PLTYP":0,"MATNR":0,"VRKME":0,"D
ATBI":0,"DATAB":0,"KNUMH":0,"landing_update_timestamp":0,"pre_bronze_dlt_update_tim
estamp":0,"bronze_dlt_update_timestamp":0}},Map(INSERTION_TIME -> 1738817013000000,
MIN_INSERTION_TIME -> 1738817013000000, MAX_INSERTION_TIME -> 1738817013000000,
OPTIMIZE_TARGET_SIZE -> 67108864),null,None,None,None))))))
25/02/06 06:01:24 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:24 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:01:25 INFO DeltaLog: Loading version 136.
25/02/06 06:01:25 INFO SnapshotEdge: [tableId=b15574a2-cdfd-4d43-b627-895cf0511cd5]
Created snapshot SnapshotEdge(path=s3://databricks-workspace-stack-f2c65-metastore-
bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-
3fb3b9160fa4/_delta_log, version=136, metadata=Metadata(dafba0e0-4a00-4017-bfec-
906a6827d303,__materialization_mat_bpp_konp_1,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"MANDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KOPOS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KAPPL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSCHL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"STFKZ","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KZBZG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSTBM","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"KONMS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSTBW","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KONWS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KRECH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KBETR","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KONWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KPEIN","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"KMEIN","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"PRSCH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KUMZA","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"KUMNE","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"MEINS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MXWRT","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"GKWRT","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"PKWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"FKWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"RSWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KWAEH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"UKBAS","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KZNEP","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KUNNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"LIFNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MWSK1","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"LOEVM_KO","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ZAEHK_IND","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"BOMAT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KBRUE","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KSPAE","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"BOSTA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_PI","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_AG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_SQ","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VALTG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VALDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ZTERM","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ANZAUF","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MIKBAS","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"MXKBAS","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"KOMXWRT","type":"decimal(13,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KLF_STG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KLF_KAL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VKKAL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"AKTNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_BO","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MWSK2","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VERTT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VERTN","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VBEWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MDFLG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KFRST","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"UASTA","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECRTT","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECRTN","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECEWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}}]},List(),Map(pipeline_internal.catalogType -> UNITY_CATALOG,
delta.enableChangeDataFeed -> true, pipelines.pipelineId -> e257d1d5-044f-4a2c-
966d-25cce6bd9aa1, pipelines.metastore.tableName ->
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`,
spark.internal.materialized_view.parentTable ->
`prod_mena`.`pre_bronze`.`bpp_konp`),Some(1728322521396)),
logSegment=LogSegment(s3://databricks-workspace-stack-f2c65-metastore-bucket/
7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log,136,WrappedArray(S3AFileStatus{path=s3://databricks-workspace-stack-
f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-
461a-870d-3fb3b9160fa4/_delta_log/00000000000000000000.json; isDirectory=false;
length=862; replication=1; blocksize=67108864; modification_time=1728322522000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e396831a08ff85d5c2841f3137c46868 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000001.00000000000000000050.compacted.json;
isDirectory=false; length=65115; replication=1; blocksize=67108864;
modification_time=1732242924000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=fb16faf9e2e48f1903af83abf020c387 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000051.00000000000000000100.compacted.json;
isDirectory=false; length=60834; replication=1; blocksize=67108864;
modification_time=1736131036000; access_time=0; owner=; group=; permission=rw-rw-
rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=ef263ab9ceb506108d8b759ab8726da8 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000101.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736217320000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=46b9843fa60890070a6b21addb25f88c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000102.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736219264000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=70dd4dba027bb4a7f4ac38650a69dafc-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000103.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736303655000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b0eb8415b7237824399b3193a4b37b77-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000104.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736390196000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b0c3dd208a6f584e05433017ea69544b-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000105.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736476655000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=da2555052d53f110a46d82a03f424ec9-1
versionId=null, S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-
metastore-bucket/7dc1e49c-2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-
870d-3fb3b9160fa4/_delta_log/00000000000000000106.json; isDirectory=false;
length=7323; replication=1; blocksize=67108864; modification_time=1736510048000;
access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=627c261c4ec62b5fb4fa4e9fa3cd5259-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000107.json; isDirectory=false; length=7322;
replication=1; blocksize=67108864; modification_time=1736562753000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=20abe1420e60d45f0d314ee339f5cdbe-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000108.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736649347000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=ac0dcc5dda9e807c6138bda06c924a85-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000109.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736735654000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=dff2853a3a324e4edff6d95dfa18b237-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000110.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736822086000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b3d20256556656b1cc2aa42d5b4add2e-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000111.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736908719000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c244da8953068c2761bdab2b21232fe3-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000112.json; isDirectory=false; length=7075;
replication=1; blocksize=67108864; modification_time=1736993292000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=cb61924d7c1e5f52f33a2e1ca899550a-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000113.json; isDirectory=false; length=6981;
replication=1; blocksize=67108864; modification_time=1736993339000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=94076f2cc2084ca08afafe3755ca402c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000114.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1736995404000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c3511872ad9f61acaa0983d716b53fc0-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000115.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737081068000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=18b094c098f7fa8c496ab56b190e9724-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000116.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737167556000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f411b4170d4d3c24da8aafdbd2324a75-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000117.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737253405000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=3d2621675c32a9d3668c24c58b04f61f-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000118.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737340437000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=70d6191cf9dfe3d5b509e0972e5c75ba-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000119.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737426631000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b2105683a944731dd42c43d96544dedd-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000120.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737513263000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=bd4a79b9da15cf29a5da64667f7c5466-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000121.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737599818000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=768a9b47c1437d4ebeb1fa17fb3fbd95-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000122.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737686303000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9d68a5f2ce94dcde1087b693ea13cde4-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000123.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737773234000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=ef31bcc184d2eb5878cd52165bd0f04f-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000124.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737858700000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c95a3dbc838a8ebddfbfa840caf33c79-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000125.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1737945275000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=d0438db91b85812de8d9e84ae679f83e-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000126.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738031565000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=5b6eb4bbdf6d969e9c5e5d1c81d728dc-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000127.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738117908000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b2a746589dd11a90765727c22f9bcc32-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000128.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738204393000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=a883b78385b723829bffa300ba2b130f-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000129.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738291004000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=30cbe912a6c976220f6315274b221d95-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000130.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738377013000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=47cc30a6c04cdece25fb86f8cbdce39d-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000131.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738463563000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=aa55ffeadb1bdcceae8095ec534adf36-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000132.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738549890000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=c56e561ea5911c30cf9ca80db3ee233c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000133.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738636377000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=e956884b6345a60b6c235e2882b6fb61-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000134.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738722828000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=f056437b974c0b0e51151ae8d6dd6819-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000135.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738800259000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=9f03f7c2bd83b9bd1b21e1eb19e25c2c-1 versionId=null,
S3AFileStatus{path=s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-
2db2-42fe-91c9-1e78ab055fde/tables/9143a5fa-916c-461a-870d-3fb3b9160fa4/
_delta_log/00000000000000000136.json; isDirectory=false; length=7323;
replication=1; blocksize=67108864; modification_time=1738815160000; access_time=0;
owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=4561650842526c5c7ef3073d8fcd4a71-1
versionId=null),com.databricks.sql.transaction.tahoe.EmptyCheckpointProvider$@61a0d
d95,1738815160000), checksumOpt=Some(VersionChecksum(Some(42b15957-eb52-420b-a773-
7f0497937d59),393711317,3,None,None,1,1,Some(List(SetTransaction(com.databricks.pip
elines.execution.core.enzyme.provenance,1738815159144,Some(1738815159144)))),Some(L
ist()),Metadata(dafba0e0-4a00-4017-bfec-
906a6827d303,__materialization_mat_bpp_konp_1,null,Format(parquet,Map()),
{"type":"struct","fields":
[{"name":"MANDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KOPOS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KAPPL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSCHL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"STFKZ","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KZBZG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSTBM","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"KONMS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KSTBW","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KONWS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KRECH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KBETR","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KONWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KPEIN","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"KMEIN","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"PRSCH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KUMZA","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"KUMNE","type":"decimal(5,0)","nullable":true,"metadata":{"scale":0}},
{"name":"MEINS","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MXWRT","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"GKWRT","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"PKWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"FKWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"RSWRT","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KWAEH","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"UKBAS","type":"decimal(15,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KZNEP","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KUNNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"LIFNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MWSK1","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"LOEVM_KO","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ZAEHK_IND","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"BOMAT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KBRUE","type":"decimal(11,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KSPAE","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"BOSTA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_PI","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_AG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_SQ","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VALTG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VALDT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ZTERM","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"ANZAUF","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MIKBAS","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"MXKBAS","type":"decimal(15,3)","nullable":true,"metadata":{"scale":3}},
{"name":"KOMXWRT","type":"decimal(13,2)","nullable":true,"metadata":{"scale":2}},
{"name":"KLF_STG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KLF_KAL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VKKAL","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"AKTNR","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KNUMA_BO","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MWSK2","type":"string","nullable":true,"metadata":{"scale"
:0}},{"name":"VERTT","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VERTN","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"VBEWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"MDFLG","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"KFRST","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"UASTA","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECRTT","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECRTN","type":"string","nullable":true,"metadata":{"scale":0}},{"name":"/
BEV1/ECEWA","type":"string","nullable":true,"metadata":{"scale":0}},
{"name":"landing_update_timestamp","type":"timestamp","nullable":true,"metadata":
{}},
{"name":"pre_bronze_dlt_update_timestamp","type":"timestamp","nullable":true,"metad
ata":{}}]},List(),Map(pipeline_internal.catalogType -> UNITY_CATALOG,
delta.enableChangeDataFeed -> true, pipelines.pipelineId -> e257d1d5-044f-4a2c-
966d-25cce6bd9aa1, pipelines.metastore.tableName ->
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`,
spark.internal.materialized_view.parentTable ->
`prod_mena`.`pre_bronze`.`bpp_konp`),Some(1728322521396)),Protocol(1,4),Some(FileSi
zeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576,
2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128,
33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080,
92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728,
138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592,
218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104,
335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616,
452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128,
570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232,
939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280,
1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648,
2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840,
4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472,
274877906944),[J@443dec8c,[J@edf249e)),None,Some(List(AddFile(part-00002-c759be10-
ab71-4052-9099-c5a07167e045-
c000.snappy.parquet,Map(),129191773,1738815155000,false,
{"numRecords":20992977,"minValues":
{"MANDT":"300","KNUMH":"0000000002","KOPOS":"01","KAPPL":"CS","KSCHL":"A-
Z2","KNUMT":"","STFKZ":"","KZBZG":"","KSTBM":0.000,"KONMS":"","KSTBW":0.00,"KONWS":
"","KRECH":"A","KBETR":-
30000.00,"KONWA":"","KPEIN":0,"KMEIN":"","PRSCH":"","KUMZA":0,"KUMNE":0,"MEINS":"",
"MXWRT":-750.00,"GKWRT":-
750.00,"PKWRT":0.00,"FKWRT":0.00,"RSWRT":0.00,"KWAEH":"","UKBAS":0.00,"KZNEP":"","K
UNNR":"","LIFNR":"","MWSK1":""},"maxValues":
{"MANDT":"300","KNUMH":"0064578864","KOPOS":"02","KAPPL":"V","KSCHL":"ZWVS","KNUMT"
:"","STFKZ":"A","KZBZG":"C","KSTBM":0.000,"KONMS":"ST","KSTBW":0.00,"KONWS":"USD","
KRECH":"I","KBETR":78853.95,"KONWA":"USD","KPEIN":240,"KMEIN":"YTR","PRSCH":"","KUM
ZA":1,"KUMNE":1,"MEINS":"YTR","MXWRT":1195.20,"GKWRT":999999999.99,"PKWRT":0.00,"FK
WRT":0.00,"RSWRT":0.00,"KWAEH":"USD","UKBAS":0.00,"KZNEP":"X","KUNNR":"H0018","LIFN
R":"0003200077","MWSK1":"Y9"},"nullCount":
{"MANDT":0,"KNUMH":0,"KOPOS":0,"KAPPL":0,"KSCHL":0,"KNUMT":0,"STFKZ":0,"KZBZG":0,"K
STBM":0,"KONMS":0,"KSTBW":0,"KONWS":0,"KRECH":0,"KBETR":0,"KONWA":0,"KPEIN":0,"KMEI
N":0,"PRSCH":0,"KUMZA":0,"KUMNE":0,"MEINS":0,"MXWRT":0,"GKWRT":0,"PKWRT":0,"FKWRT":
0,"RSWRT":0,"KWAEH":0,"UKBAS":0,"KZNEP":0,"KUNNR":0,"LIFNR":0,"MWSK1":0}},Map(INSER
TION_TIME -> 1738815003000002, MIN_INSERTION_TIME -> 1738815003000002,
MAX_INSERTION_TIME -> 1738815003000002, OPTIMIZE_TARGET_SIZE ->
67108864),null,None,None,None), AddFile(part-00001-e26cfcae-b5a7-4c0f-9a9e-
f54642084b53-c000.snappy.parquet,Map(),132210771,1738815003000,false,
{"numRecords":21630100,"minValues":
{"MANDT":"300","KNUMH":"0000118085","KOPOS":"01","KAPPL":"M","KSCHL":"PB00","KNUMT"
:"","STFKZ":"A","KZBZG":"","KSTBM":0.000,"KONMS":"","KSTBW":0.00,"KONWS":"","KRECH"
:"C","KBETR":0.00,"KONWA":"","KPEIN":1,"KMEIN":"","PRSCH":"","KUMZA":0,"KUMNE":0,"M
EINS":"","MXWRT":0.00,"GKWRT":0.00,"PKWRT":0.00,"FKWRT":0.00,"RSWRT":0.00,"KWAEH":"
","UKBAS":0.00,"KZNEP":"","KUNNR":"","LIFNR":"","MWSK1":""},"maxValues":
{"MANDT":"300","KNUMH":"0064763403","KOPOS":"04","KAPPL":"V","KSCHL":"ZVKP","KNUMT"
:"","STFKZ":"A","KZBZG":"C","KSTBM":0.000,"KONMS":"YTR","KSTBW":0.00,"KONWS":"AED",
"KRECH":"C","KBETR":115563540.00,"KONWA":"ZAR","KPEIN":1,"KMEIN":"YTR","PRSCH":"","
KUMZA":1,"KUMNE":15,"MEINS":"YTR","MXWRT":0.00,"GKWRT":0.00,"PKWRT":0.00,"FKWRT":0.
00,"RSWRT":0.00,"KWAEH":"SAR","UKBAS":0.00,"KZNEP":"X","KUNNR":"","LIFNR":"00032000
77","MWSK1":""},"nullCount":
{"MANDT":0,"KNUMH":0,"KOPOS":0,"KAPPL":0,"KSCHL":0,"KNUMT":0,"STFKZ":0,"KZBZG":0,"K
STBM":0,"KONMS":0,"KSTBW":0,"KONWS":0,"KRECH":0,"KBETR":0,"KONWA":0,"KPEIN":0,"KMEI
N":0,"PRSCH":0,"KUMZA":0,"KUMNE":0,"MEINS":0,"MXWRT":0,"GKWRT":0,"PKWRT":0,"FKWRT":
0,"RSWRT":0,"KWAEH":0,"UKBAS":0,"KZNEP":0,"KUNNR":0,"LIFNR":0,"MWSK1":0}},Map(INSER
TION_TIME -> 1738815003000001, MIN_INSERTION_TIME -> 1738815003000001,
MAX_INSERTION_TIME -> 1738815003000001, OPTIMIZE_TARGET_SIZE ->
67108864),null,None,None,None), AddFile(part-00000-ca50ee68-36e5-49f0-a237-
9310c7778b5e-c000.snappy.parquet,Map(),132308773,1738815024000,false,
{"numRecords":22310100,"minValues":
{"MANDT":"300","KNUMH":"0000103639","KOPOS":"01","KAPPL":"M","KSCHL":"PB00","KNUMT"
:"","STFKZ":"A","KZBZG":"","KSTBM":0.000,"KONMS":"","KSTBW":0.00,"KONWS":"","KRECH"
:"C","KBETR":0.00,"KONWA":"AED","KPEIN":1,"KMEIN":"","PRSCH":"","KUMZA":0,"KUMNE":0
,"MEINS":"","MXWRT":0.00,"GKWRT":0.00,"PKWRT":0.00,"FKWRT":0.00,"RSWRT":0.00,"KWAEH
":"","UKBAS":0.00,"KZNEP":"","KUNNR":"","LIFNR":"","MWSK1":""},"maxValues":
{"MANDT":"300","KNUMH":"0064764788","KOPOS":"01","KAPPL":"V","KSCHL":"ZVKP","KNUMT"
:"","STFKZ":"A","KZBZG":"","KSTBM":0.000,"KONMS":"ST","KSTBW":0.00,"KONWS":"AED","K
RECH":"C","KBETR":103998.95,"KONWA":"USD","KPEIN":1,"KMEIN":"YTR","PRSCH":"","KUMZA
":1,"KUMNE":1,"MEINS":"ST","MXWRT":0.00,"GKWRT":0.00,"PKWRT":0.00,"FKWRT":0.00,"RSW
RT":0.00,"KWAEH":"SAR","UKBAS":0.00,"KZNEP":"X","KUNNR":"","LIFNR":"","MWSK1":""},"
nullCount":
{"MANDT":0,"KNUMH":0,"KOPOS":0,"KAPPL":0,"KSCHL":0,"KNUMT":0,"STFKZ":0,"KZBZG":0,"K
STBM":0,"KONMS":0,"KSTBW":0,"KONWS":0,"KRECH":0,"KBETR":0,"KONWA":0,"KPEIN":0,"KMEI
N":0,"PRSCH":0,"KUMZA":0,"KUMNE":0,"MEINS":0,"MXWRT":0,"GKWRT":0,"PKWRT":0,"FKWRT":
0,"RSWRT":0,"KWAEH":0,"UKBAS":0,"KZNEP":0,"KUNNR":0,"LIFNR":0,"MWSK1":0}},Map(INSER
TION_TIME -> 1738815003000000, MIN_INSERTION_TIME -> 1738815003000000,
MAX_INSERTION_TIME -> 1738815003000000, OPTIMIZE_TARGET_SIZE ->
67108864),null,None,None,None))))))
25/02/06 06:01:25 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:25 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:25 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:25 INFO CodecPool: Got brand-new decompressor [.snappy]
25/02/06 06:01:25 INFO CodecPool: Got brand-new decompressor [.snappy]
25/02/06 06:01:26 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
27.594461 ms.
25/02/06 06:01:26 INFO HiveCheckPermissions: Computed permissions to check in
15.671853 milliseconds
25/02/06 06:01:26 INFO HiveCheckPermissions: Checking all permissions took
15.779194 milliseconds
25/02/06 06:01:27 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:27 INFO DeltaLog: Prefetch Delta checkpoint version 400 for
s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-
1e78ab055fde/tables/21a23d04-c177-49d5-8d77-18da4df94e93/_delta_log took 5080
milliseconds
25/02/06 06:01:27 INFO DeltaLog: Prefetch Delta checkpoint version 534 for
s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-
1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/_delta_log took 5126
milliseconds
25/02/06 06:01:27 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:27 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:27 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:27 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
2.941072 ms.
25/02/06 06:01:27 INFO HiveCheckPermissions: Computed permissions to check in
9.11642 milliseconds
25/02/06 06:01:27 INFO HiveCheckPermissions: Checking all permissions took 9.211848
milliseconds
25/02/06 06:01:27 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
5.961737 ms.
25/02/06 06:01:27 INFO HiveCheckPermissions: Computed permissions to check in
8.876183 milliseconds
25/02/06 06:01:27 INFO HiveCheckPermissions: Checking all permissions took 8.981454
milliseconds
25/02/06 06:01:28 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:28 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:28 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:28 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:28 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
12.690426 ms.
25/02/06 06:01:28 INFO HiveCheckPermissions: Computed permissions to check in
4.404518 milliseconds
25/02/06 06:01:28 INFO HiveCheckPermissions: Checking all permissions took 4.513916
milliseconds
25/02/06 06:01:28 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
6.265997 ms.
25/02/06 06:01:28 INFO HiveCheckPermissions: Computed permissions to check in
9.868298 milliseconds
25/02/06 06:01:28 INFO HiveCheckPermissions: Checking all permissions took 9.966334
milliseconds
25/02/06 06:01:29 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:29 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:29 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:29 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:29 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
6.522074 ms.
25/02/06 06:01:29 INFO HiveCheckPermissions: Computed permissions to check in
3.81521 milliseconds
25/02/06 06:01:29 INFO HiveCheckPermissions: Checking all permissions took 3.906758
milliseconds
25/02/06 06:01:29 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
13.34076 ms.
25/02/06 06:01:29 INFO HiveCheckPermissions: Computed permissions to check in
12.349463 milliseconds
25/02/06 06:01:29 INFO HiveCheckPermissions: Checking all permissions took 12.4443
milliseconds
25/02/06 06:01:30 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:30 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:30 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:30 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:30 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.567535 ms.
25/02/06 06:01:30 INFO HiveCheckPermissions: Computed permissions to check in
2.657499 milliseconds
25/02/06 06:01:30 INFO HiveCheckPermissions: Checking all permissions took 2.724064
milliseconds
25/02/06 06:01:30 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
6.8882 ms.
25/02/06 06:01:30 INFO HiveCheckPermissions: Computed permissions to check in
8.862464 milliseconds
25/02/06 06:01:30 INFO HiveCheckPermissions: Checking all permissions took 8.967121
milliseconds
25/02/06 06:01:31 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:31 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:31 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:31 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:31 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.993129 ms.
25/02/06 06:01:31 INFO HiveCheckPermissions: Computed permissions to check in
3.344536 milliseconds
25/02/06 06:01:31 INFO HiveCheckPermissions: Checking all permissions took 3.440852
milliseconds
25/02/06 06:01:31 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
12.067705 ms.
25/02/06 06:01:31 INFO HiveCheckPermissions: Computed permissions to check in
13.08056 milliseconds
25/02/06 06:01:31 INFO HiveCheckPermissions: Checking all permissions took
13.175655 milliseconds
25/02/06 06:01:32 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:32 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:32 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:32 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:32 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.357587 ms.
25/02/06 06:01:32 INFO HiveCheckPermissions: Computed permissions to check in
2.187179 milliseconds
25/02/06 06:01:32 INFO HiveCheckPermissions: Checking all permissions took 2.266181
milliseconds
25/02/06 06:01:32 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
9.345337 ms.
25/02/06 06:01:32 INFO HiveCheckPermissions: Computed permissions to check in
11.309399 milliseconds
25/02/06 06:01:32 INFO HiveCheckPermissions: Checking all permissions took
11.385226 milliseconds
25/02/06 06:01:33 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:33 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:33 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:33 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:33 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.997175 ms.
25/02/06 06:01:33 INFO HiveCheckPermissions: Computed permissions to check in
3.088998 milliseconds
25/02/06 06:01:33 INFO HiveCheckPermissions: Checking all permissions took 3.173248
milliseconds
25/02/06 06:01:33 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
15.952684 ms.
25/02/06 06:01:33 INFO HiveCheckPermissions: Computed permissions to check in
12.047528 milliseconds
25/02/06 06:01:33 INFO HiveCheckPermissions: Checking all permissions took
12.125538 milliseconds
25/02/06 06:01:34 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:34 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:34 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:34 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:34 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
2.244545 ms.
25/02/06 06:01:34 INFO HiveCheckPermissions: Computed permissions to check in
3.297236 milliseconds
25/02/06 06:01:34 INFO HiveCheckPermissions: Checking all permissions took 3.381364
milliseconds
25/02/06 06:01:34 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
14.66728 ms.
25/02/06 06:01:34 INFO HiveCheckPermissions: Computed permissions to check in
18.175238 milliseconds
25/02/06 06:01:34 INFO HiveCheckPermissions: Checking all permissions took
18.283451 milliseconds
25/02/06 06:01:35 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:35 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:35 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:35 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:35 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.348159 ms.
25/02/06 06:01:35 INFO HiveCheckPermissions: Computed permissions to check in
9.672312 milliseconds
25/02/06 06:01:35 INFO HiveCheckPermissions: Checking all permissions took 9.76852
milliseconds
25/02/06 06:01:35 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
15.310284 ms.
25/02/06 06:01:35 INFO HiveCheckPermissions: Computed permissions to check in
15.998724 milliseconds
25/02/06 06:01:35 INFO HiveCheckPermissions: Checking all permissions took
16.082342 milliseconds
25/02/06 06:01:36 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:36 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:36 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:36 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:36 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
2.311751 ms.
25/02/06 06:01:36 INFO HiveCheckPermissions: Computed permissions to check in
2.909778 milliseconds
25/02/06 06:01:36 INFO HiveCheckPermissions: Checking all permissions took 2.996903
milliseconds
25/02/06 06:01:36 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
19.054465 ms.
25/02/06 06:01:36 INFO HiveCheckPermissions: Computed permissions to check in
15.212497 milliseconds
25/02/06 06:01:36 INFO HiveCheckPermissions: Checking all permissions took
15.291828 milliseconds
25/02/06 06:01:37 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-
20250206055605-0000/4 is now DECOMMISSIONED (worker decommissioned because of kill
request from HTTP endpoint (data migration disabled))
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Asked to decommission executor
app-20250206055605-0000/4
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Decommission executors: 4
25/02/06 06:01:37 INFO BlockManagerMasterEndpoint: Mark BlockManagers
(BlockManagerId(4, 10.183.231.236, 41737, None)) as being decommissioning.
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Notify executor 4 to
decommission.
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Executor app-20250206055605-
0000/4 decommissioned: ExecutorDecommissionInfo(worker decommissioned because of
kill request from HTTP endpoint (data migration
disabled),Some(10.183.231.236),false,None)
25/02/06 06:01:37 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-
20250206055605-0000/4 is now LOST (worker lost: kill request from HTTP endpoint)
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Executor app-20250206055605-
0000/4 removed: worker lost: kill request from HTTP endpoint
25/02/06 06:01:37 INFO StandaloneAppClient$ClientEndpoint: Master removed worker
worker-20250206055543-10.183.231.236-43141: kill request from HTTP endpoint
25/02/06 06:01:37 INFO StandaloneSchedulerBackend: Worker worker-20250206055543-
10.183.231.236-43141 removed: kill request from HTTP endpoint
25/02/06 06:01:37 INFO TaskSchedulerImpl: Executor 4 on 10.183.231.236 is
decommissioned after 2 ms.
25/02/06 06:01:37 INFO DAGScheduler: Executor lost: 4 (epoch 5)
25/02/06 06:01:37 INFO TaskSchedulerImpl: Handle removed worker worker-
20250206055543-10.183.231.236-43141: kill request from HTTP endpoint
25/02/06 06:01:37 INFO BlockManagerMasterEndpoint: Trying to remove executor 4 from
BlockManagerMaster.
25/02/06 06:01:37 INFO BlockManagerMasterEndpoint: Removing block manager
BlockManagerId(4, 10.183.231.236, 41737, None)
25/02/06 06:01:37 INFO BlockManagerMaster: Removed 4 successfully in removeExecutor
25/02/06 06:01:37 INFO DAGScheduler: Shuffle files lost for host: 10.183.231.236
(epoch 5)
25/02/06 06:01:37 INFO DAGScheduler: Shuffle files lost for worker worker-
20250206055543-10.183.231.236-43141 on host 10.183.231.236
25/02/06 06:01:37 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:37 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:37 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:37 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:37 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.320725 ms.
25/02/06 06:01:37 INFO HiveCheckPermissions: Computed permissions to check in
1.954004 milliseconds
25/02/06 06:01:37 INFO HiveCheckPermissions: Checking all permissions took 2.017409
milliseconds
25/02/06 06:01:37 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
17.644374 ms.
25/02/06 06:01:37 INFO HiveCheckPermissions: Computed permissions to check in
16.894242 milliseconds
25/02/06 06:01:37 INFO HiveCheckPermissions: Checking all permissions took 16.97801
milliseconds
25/02/06 06:01:38 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:38 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:38 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:38 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:38 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.278688 ms.
25/02/06 06:01:38 INFO HiveCheckPermissions: Computed permissions to check in
1.89117 milliseconds
25/02/06 06:01:38 INFO HiveCheckPermissions: Checking all permissions took 1.963836
milliseconds
25/02/06 06:01:38 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
16.45017 ms.
25/02/06 06:01:38 INFO HiveCheckPermissions: Computed permissions to check in
15.639673 milliseconds
25/02/06 06:01:38 INFO HiveCheckPermissions: Checking all permissions took
15.729756 milliseconds
25/02/06 06:01:39 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:39 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:39 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:39 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:39 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.335142 ms.
25/02/06 06:01:39 INFO HiveCheckPermissions: Computed permissions to check in
1.942044 milliseconds
25/02/06 06:01:39 INFO HiveCheckPermissions: Checking all permissions took 2.009899
milliseconds
25/02/06 06:01:39 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
18.612952 ms.
25/02/06 06:01:39 INFO HiveCheckPermissions: Computed permissions to check in
17.994249 milliseconds
25/02/06 06:01:39 INFO HiveCheckPermissions: Checking all permissions took
18.069701 milliseconds
25/02/06 06:01:40 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:40 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:40 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:40 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:40 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.545029 ms.
25/02/06 06:01:40 INFO HiveCheckPermissions: Computed permissions to check in
2.233545 milliseconds
25/02/06 06:01:40 INFO HiveCheckPermissions: Checking all permissions took 2.291594
milliseconds
25/02/06 06:01:40 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
21.421746 ms.
25/02/06 06:01:40 INFO HiveCheckPermissions: Computed permissions to check in
20.337384 milliseconds
25/02/06 06:01:40 INFO HiveCheckPermissions: Checking all permissions took
20.414589 milliseconds
25/02/06 06:01:41 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:41 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:41 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:41 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:41 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.318638 ms.
25/02/06 06:01:41 INFO HiveCheckPermissions: Computed permissions to check in
1.876597 milliseconds
25/02/06 06:01:41 INFO HiveCheckPermissions: Checking all permissions took 1.937889
milliseconds
25/02/06 06:01:41 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
22.743404 ms.
25/02/06 06:01:41 INFO HiveCheckPermissions: Computed permissions to check in
22.280907 milliseconds
25/02/06 06:01:41 INFO HiveCheckPermissions: Checking all permissions took 22.3694
milliseconds
25/02/06 06:01:42 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:42 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:42 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:42 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:42 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.333773 ms.
25/02/06 06:01:42 INFO HiveCheckPermissions: Computed permissions to check in
1.929605 milliseconds
25/02/06 06:01:42 INFO HiveCheckPermissions: Checking all permissions took 1.995541
milliseconds
25/02/06 06:01:42 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
25.154168 ms.
25/02/06 06:01:42 INFO HiveCheckPermissions: Computed permissions to check in
23.619884 milliseconds
25/02/06 06:01:42 INFO HiveCheckPermissions: Checking all permissions took
23.705052 milliseconds
25/02/06 06:01:43 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:43 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:43 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:43 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:43 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.665507 ms.
25/02/06 06:01:43 INFO HiveCheckPermissions: Computed permissions to check in
2.006067 milliseconds
25/02/06 06:01:43 INFO HiveCheckPermissions: Checking all permissions took 2.089476
milliseconds
25/02/06 06:01:43 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
26.062834 ms.
25/02/06 06:01:43 INFO HiveCheckPermissions: Computed permissions to check in
24.71639 milliseconds
25/02/06 06:01:43 INFO HiveCheckPermissions: Checking all permissions took
24.829451 milliseconds
25/02/06 06:01:44 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:44 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:44 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:44 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:44 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.251754 ms.
25/02/06 06:01:44 INFO HiveCheckPermissions: Computed permissions to check in
2.008834 milliseconds
25/02/06 06:01:44 INFO HiveCheckPermissions: Checking all permissions took 2.084883
milliseconds
25/02/06 06:01:45 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
28.25762 ms.
25/02/06 06:01:45 INFO HiveCheckPermissions: Computed permissions to check in
26.354209 milliseconds
25/02/06 06:01:45 INFO HiveCheckPermissions: Checking all permissions took
26.450078 milliseconds
25/02/06 06:01:45 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:45 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:45 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:45 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:45 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.310975 ms.
25/02/06 06:01:45 INFO HiveCheckPermissions: Computed permissions to check in
1.864559 milliseconds
25/02/06 06:01:45 INFO HiveCheckPermissions: Checking all permissions took 1.922232
milliseconds
25/02/06 06:01:46 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
29.810229 ms.
25/02/06 06:01:46 INFO HiveCheckPermissions: Computed permissions to check in
28.939405 milliseconds
25/02/06 06:01:46 INFO HiveCheckPermissions: Checking all permissions took
29.025471 milliseconds
25/02/06 06:01:46 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:46 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:46 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:46 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:46 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
2.049287 ms.
25/02/06 06:01:46 INFO HiveCheckPermissions: Computed permissions to check in
2.837078 milliseconds
25/02/06 06:01:46 INFO HiveCheckPermissions: Checking all permissions took 2.924794
milliseconds
25/02/06 06:01:47 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
28.309862 ms.
25/02/06 06:01:47 INFO HiveCheckPermissions: Computed permissions to check in
25.987681 milliseconds
25/02/06 06:01:47 INFO HiveCheckPermissions: Checking all permissions took
26.069321 milliseconds
25/02/06 06:01:47 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:47 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:47 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:47 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:47 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.285219 ms.
25/02/06 06:01:47 INFO HiveCheckPermissions: Computed permissions to check in
2.016296 milliseconds
25/02/06 06:01:47 INFO HiveCheckPermissions: Checking all permissions took 2.083132
milliseconds
25/02/06 06:01:48 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
49.627789 ms.
25/02/06 06:01:48 INFO HiveCheckPermissions: Computed permissions to check in
42.888199 milliseconds
25/02/06 06:01:48 INFO HiveCheckPermissions: Checking all permissions took
42.971662 milliseconds
25/02/06 06:01:48 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:49 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:49 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:49 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:49 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.321975 ms.
25/02/06 06:01:49 INFO HiveCheckPermissions: Computed permissions to check in
1.892754 milliseconds
25/02/06 06:01:49 INFO HiveCheckPermissions: Checking all permissions took 1.953376
milliseconds
25/02/06 06:01:49 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
33.268839 ms.
25/02/06 06:01:49 INFO HiveCheckPermissions: Computed permissions to check in
30.875797 milliseconds
25/02/06 06:01:49 INFO HiveCheckPermissions: Checking all permissions took
30.956848 milliseconds
25/02/06 06:01:50 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:50 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:50 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:50 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:50 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.305826 ms.
25/02/06 06:01:50 INFO HiveCheckPermissions: Computed permissions to check in
1.836279 milliseconds
25/02/06 06:01:50 INFO HiveCheckPermissions: Checking all permissions took 1.919548
milliseconds
25/02/06 06:01:50 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
36.771804 ms.
25/02/06 06:01:50 INFO HiveCheckPermissions: Computed permissions to check in
33.690933 milliseconds
25/02/06 06:01:50 INFO HiveCheckPermissions: Checking all permissions took 33.76893
milliseconds
25/02/06 06:01:51 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:51 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:51 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:51 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:51 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.342632 ms.
25/02/06 06:01:51 INFO HiveCheckPermissions: Computed permissions to check in
1.8546 milliseconds
25/02/06 06:01:51 INFO HiveCheckPermissions: Checking all permissions took 1.914975
milliseconds
25/02/06 06:01:52 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
36.391193 ms.
25/02/06 06:01:52 INFO HiveCheckPermissions: Computed permissions to check in
33.445299 milliseconds
25/02/06 06:01:52 INFO HiveCheckPermissions: Checking all permissions took
33.535722 milliseconds
25/02/06 06:01:52 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:52 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:52 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:52 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:52 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.96032 ms.
25/02/06 06:01:52 INFO HiveCheckPermissions: Computed permissions to check in
2.230516 milliseconds
25/02/06 06:01:52 INFO HiveCheckPermissions: Checking all permissions took 2.290278
milliseconds
25/02/06 06:01:53 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
38.258648 ms.
25/02/06 06:01:53 INFO HiveCheckPermissions: Computed permissions to check in
35.2854 milliseconds
25/02/06 06:01:53 INFO HiveCheckPermissions: Checking all permissions took
35.373516 milliseconds
25/02/06 06:01:53 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:53 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:53 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:53 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:53 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.26952 ms.
25/02/06 06:01:53 INFO HiveCheckPermissions: Computed permissions to check in
1.844829 milliseconds
25/02/06 06:01:53 INFO HiveCheckPermissions: Checking all permissions took 1.905904
milliseconds
25/02/06 06:01:54 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
40.330107 ms.
25/02/06 06:01:54 INFO HiveCheckPermissions: Computed permissions to check in
36.69517 milliseconds
25/02/06 06:01:54 INFO HiveCheckPermissions: Checking all permissions took
36.782772 milliseconds
25/02/06 06:01:55 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:55 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:55 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:55 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:55 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.339515 ms.
25/02/06 06:01:55 INFO HiveCheckPermissions: Computed permissions to check in
1.860042 milliseconds
25/02/06 06:01:55 INFO HiveCheckPermissions: Checking all permissions took 1.917639
milliseconds
25/02/06 06:01:55 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
42.900131 ms.
25/02/06 06:01:55 INFO HiveCheckPermissions: Computed permissions to check in
37.65766 milliseconds
25/02/06 06:01:55 INFO HiveCheckPermissions: Checking all permissions took
37.742927 milliseconds
25/02/06 06:01:56 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:56 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:56 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:56 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:56 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.236943 ms.
25/02/06 06:01:56 INFO HiveCheckPermissions: Computed permissions to check in
1.853209 milliseconds
25/02/06 06:01:56 INFO HiveCheckPermissions: Checking all permissions took 1.913832
milliseconds
25/02/06 06:01:57 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
42.860329 ms.
25/02/06 06:01:57 INFO HiveCheckPermissions: Computed permissions to check in
45.328398 milliseconds
25/02/06 06:01:57 INFO HiveCheckPermissions: Checking all permissions took
45.423718 milliseconds
25/02/06 06:01:57 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_2ba61ec2_c859_4b46_ab90_5e6b1
e15c4f3`.`__materialization_mat_bpp_a155_1`
25/02/06 06:01:57 INFO DeltaTableV2: Updating catalog for table
`__databricks_internal`.`__dlt_materialization_schema_e257d1d5_044f_4a2c_966d_25cce
6bd9aa1`.`__materialization_mat_bpp_konp_1`
25/02/06 06:01:57 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_azsd_ald062_bpp_azsd_ald052`
25/02/06 06:01:57 INFO DeltaTableV2: Updating catalog for table
`prod_mena`.`silver`.`bpp_pmaterial_bpp_tmaterial`
25/02/06 06:01:57 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
1.312306 ms.
25/02/06 06:01:57 INFO HiveCheckPermissions: Computed permissions to check in
1.862375 milliseconds
25/02/06 06:01:57 INFO HiveCheckPermissions: Checking all permissions took 1.922823
milliseconds
25/02/06 06:01:58 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
43.546935 ms.
25/02/06 06:01:58 INFO HiveCheckPermissions: Computed permissions to check in
39.698088 milliseconds
25/02/06 06:01:58 INFO HiveCheckPermissions: Checking all permissions took
39.783347 milliseconds
25/02/06 06:01:59 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
44.681772 ms.
25/02/06 06:01:59 INFO HiveCheckPermissions: Computed permissions to check in
41.151721 milliseconds
25/02/06 06:01:59 INFO HiveCheckPermissions: Checking all permissions took
41.241392 milliseconds
25/02/06 06:02:00 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
46.278279 ms.
25/02/06 06:02:00 INFO HiveCheckPermissions: Computed permissions to check in
40.889412 milliseconds
25/02/06 06:02:00 INFO HiveCheckPermissions: Checking all permissions took
40.974498 milliseconds
25/02/06 06:02:00 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
47.201804 ms.
25/02/06 06:02:00 INFO HiveCheckPermissions: Computed permissions to check in
40.704154 milliseconds
25/02/06 06:02:00 INFO HiveCheckPermissions: Checking all permissions took
40.783419 milliseconds
25/02/06 06:02:00 INFO ProgressReporter$: Removed result fetcher for
5400065285585087590_7337167144654976902_ef479873a51d419caffb2e0977d76b48
25/02/06 06:02:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:02:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:03:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:03:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:04:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:04:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:05:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:05:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:06:00 INFO DataSourceFactory$: DataSource Jdbc URL:
jdbc:mariadb://md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com:3306/
organization8517174019523704?
useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertif
icate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt
25/02/06 06:06:00 INFO HikariDataSource: metastore-monitor - Starting...
25/02/06 06:06:00 INFO HikariDataSource: metastore-monitor - Start completed.
25/02/06 06:06:00 INFO HikariDataSource: metastore-monitor - Shutdown initiated...
25/02/06 06:06:00 INFO HikariDataSource: metastore-monitor - Shutdown completed.
25/02/06 06:06:00 INFO MetastoreMonitor: Metastore healthcheck successful
(connection duration = 304 milliseconds)
25/02/06 06:06:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:06:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:06:23 INFO DriverCorral: DBFS health check ok
25/02/06 06:06:23 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:06:23 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:06:23 INFO DriverCorral: Metastore health check ok
25/02/06 06:07:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:07:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:07:33 INFO ProgressReporter$: Added result fetcher for
5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9
25/02/06 06:07:33 INFO ProgressReporter$: Reporting progress for running commands:
5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9 [13
occurrences]
25/02/06 06:07:34 INFO ProgressReporter$: Reporting partial results for running
commands: 5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9
[149 occurrences]
25/02/06 06:07:34 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
41.275405 ms.
25/02/06 06:07:34 INFO HiveCheckPermissions: Computed permissions to check in
38.044679 milliseconds
25/02/06 06:07:34 INFO HiveCheckPermissions: Checking all permissions took
38.119305 milliseconds
25/02/06 06:07:35 INFO HiveUnityCatalogCheckRule: Collecting SQL actions took
43.612598 ms.
25/02/06 06:07:35 INFO HiveCheckPermissions: Computed permissions to check in
41.262015 milliseconds
25/02/06 06:07:35 INFO HiveCheckPermissions: Checking all permissions took
41.338692 milliseconds
25/02/06 06:07:35 INFO ClusterLoadMonitor: Added query with execution ID:32.
Current active queries:1
25/02/06 06:07:35 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 12, Total cores: 8, Current load: 1, Current Avg
load: 0, New parallelism: 8
25/02/06 06:07:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0,
New Ema: 1.0
25/02/06 06:07:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:40 INFO AdviceStore: Added advice USE_RANGE_JOIN_ADVICE (version 3)
for executionId 32, jobGroupId
5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9: Use range
join optimization This query has a join condition that can benefit from range join
optimization. To improve performance, consider adding a range join hint. Learn more
(optimizations/range-join.html)
25/02/06 06:07:41 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:07:41 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:07:41 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:07:41 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet,
numFilesInSegment: 1, totalFileSize: 215195)
25/02/06 06:07:41 INFO AWSCredentialProviderList:V3: Using credentials for bucket
databricks-workspace-stack-f2c65-metastore-bucket from
com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider
25/02/06 06:07:42 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON,
numFilesInSegment: 13, totalFileSize: 7231343)
25/02/06 06:07:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:43 INFO SnapshotEdge: [tableId=22d22904-1726-434e-b13a-2d5519186a2d]
Populate SmallTableCache with prefetched checkpoint of version534 for table
s3://databricks-workspace-stack-f2c65-metastore-bucket/7dc1e49c-2db2-42fe-91c9-
1e78ab055fde/tables/428f15be-7119-4417-a35d-5e2c2c9a7a9e/_delta_log
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#421),MATL_CAT#461 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#295583),isnotnull(PLTYP#295586),isnotnull(MATNR#295587),isnotnull(V
TWEG#295585),isnotnull(VKORG#295584),(KSCHL#295583 = VKP0),(PLTYP#295586 = OR),
(gettimestamp(DATBI#295589, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-01 00:00:00),(gettimestamp(DATAB#295590, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-02 00:00:00),VTWEG#295585 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#295596)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#295663),isnotnull(PLTYP#295666),isnotnull(MATNR#295667),isnotnull(V
TWEG#295665),isnotnull(VKORG#295664),(KSCHL#295663 = VKP0),(PLTYP#295666 = CR),
(gettimestamp(DATBI#295669, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-01 00:00:00),(gettimestamp(DATAB#295670, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-02 00:00:00),VTWEG#295665 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#295676)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#295754),isnotnull(PLTYP#295757),isnotnull(MATNR#295758),isnotnull(V
TWEG#295756),isnotnull(VKORG#295755),(KSCHL#295754 = VKP0),(PLTYP#295757 = SL),
(gettimestamp(DATBI#295760, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-01 00:00:00),(gettimestamp(DATAB#295761, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-02 00:00:00),VTWEG#295756 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#295767)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#1610),MATL_CAT#1650 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296005),isnotnull(PLTYP#296008),isnotnull(MATNR#296009),isnotnull(V
TWEG#296007),isnotnull(VKORG#296006),(KSCHL#296005 = VKP0),(PLTYP#296008 = OR),
(gettimestamp(DATBI#296011, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-02 00:00:00),(gettimestamp(DATAB#296012, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-03 00:00:00),VTWEG#296007 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296018)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296085),isnotnull(PLTYP#296088),isnotnull(MATNR#296089),isnotnull(V
TWEG#296087),isnotnull(VKORG#296086),(KSCHL#296085 = VKP0),(PLTYP#296088 = CR),
(gettimestamp(DATBI#296091, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-02 00:00:00),(gettimestamp(DATAB#296092, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-03 00:00:00),VTWEG#296087 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296098)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296176),isnotnull(PLTYP#296179),isnotnull(MATNR#296180),isnotnull(V
TWEG#296178),isnotnull(VKORG#296177),(KSCHL#296176 = VKP0),(PLTYP#296179 = SL),
(gettimestamp(DATBI#296182, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-02 00:00:00),(gettimestamp(DATAB#296183, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-03 00:00:00),VTWEG#296178 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296189)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#3659),MATL_CAT#3699 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296427),isnotnull(PLTYP#296430),isnotnull(MATNR#296431),isnotnull(V
TWEG#296429),isnotnull(VKORG#296428),(KSCHL#296427 = VKP0),(PLTYP#296430 = OR),
(gettimestamp(DATBI#296433, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-03 00:00:00),(gettimestamp(DATAB#296434, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-04 00:00:00),VTWEG#296429 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296440)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296507),isnotnull(PLTYP#296510),isnotnull(MATNR#296511),isnotnull(V
TWEG#296509),isnotnull(VKORG#296508),(KSCHL#296507 = VKP0),(PLTYP#296510 = CR),
(gettimestamp(DATBI#296513, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-03 00:00:00),(gettimestamp(DATAB#296514, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-04 00:00:00),VTWEG#296509 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296520)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296598),isnotnull(PLTYP#296601),isnotnull(MATNR#296602),isnotnull(V
TWEG#296600),isnotnull(VKORG#296599),(KSCHL#296598 = VKP0),(PLTYP#296601 = SL),
(gettimestamp(DATBI#296604, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-03 00:00:00),(gettimestamp(DATAB#296605, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-04 00:00:00),VTWEG#296600 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296611)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#6130),MATL_CAT#6170 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296849),isnotnull(PLTYP#296852),isnotnull(MATNR#296853),isnotnull(V
TWEG#296851),isnotnull(VKORG#296850),(KSCHL#296849 = VKP0),(PLTYP#296852 = OR),
(gettimestamp(DATBI#296855, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-04 00:00:00),(gettimestamp(DATAB#296856, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-05 00:00:00),VTWEG#296851 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296862)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#296929),isnotnull(PLTYP#296932),isnotnull(MATNR#296933),isnotnull(V
TWEG#296931),isnotnull(VKORG#296930),(KSCHL#296929 = VKP0),(PLTYP#296932 = CR),
(gettimestamp(DATBI#296935, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-04 00:00:00),(gettimestamp(DATAB#296936, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-05 00:00:00),VTWEG#296931 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#296942)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297020),isnotnull(PLTYP#297023),isnotnull(MATNR#297024),isnotnull(V
TWEG#297022),isnotnull(VKORG#297021),(KSCHL#297020 = VKP0),(PLTYP#297023 = SL),
(gettimestamp(DATBI#297026, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-04 00:00:00),(gettimestamp(DATAB#297027, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-05 00:00:00),VTWEG#297022 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297033)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#9023),MATL_CAT#9063 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297271),isnotnull(PLTYP#297274),isnotnull(MATNR#297275),isnotnull(V
TWEG#297273),isnotnull(VKORG#297272),(KSCHL#297271 = VKP0),(PLTYP#297274 = OR),
(gettimestamp(DATBI#297277, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-05 00:00:00),(gettimestamp(DATAB#297278, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-06 00:00:00),VTWEG#297273 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297284)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297351),isnotnull(PLTYP#297354),isnotnull(MATNR#297355),isnotnull(V
TWEG#297353),isnotnull(VKORG#297352),(KSCHL#297351 = VKP0),(PLTYP#297354 = CR),
(gettimestamp(DATBI#297357, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-05 00:00:00),(gettimestamp(DATAB#297358, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-06 00:00:00),VTWEG#297353 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297364)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297442),isnotnull(PLTYP#297445),isnotnull(MATNR#297446),isnotnull(V
TWEG#297444),isnotnull(VKORG#297443),(KSCHL#297442 = VKP0),(PLTYP#297445 = SL),
(gettimestamp(DATBI#297448, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-05 00:00:00),(gettimestamp(DATAB#297449, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-06 00:00:00),VTWEG#297444 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297455)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#12338),MATL_CAT#12378 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297693),isnotnull(PLTYP#297696),isnotnull(MATNR#297697),isnotnull(V
TWEG#297695),isnotnull(VKORG#297694),(KSCHL#297693 = VKP0),(PLTYP#297696 = OR),
(gettimestamp(DATBI#297699, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-06 00:00:00),(gettimestamp(DATAB#297700, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-07 00:00:00),VTWEG#297695 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297706)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297773),isnotnull(PLTYP#297776),isnotnull(MATNR#297777),isnotnull(V
TWEG#297775),isnotnull(VKORG#297774),(KSCHL#297773 = VKP0),(PLTYP#297776 = CR),
(gettimestamp(DATBI#297779, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-06 00:00:00),(gettimestamp(DATAB#297780, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-07 00:00:00),VTWEG#297775 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297786)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#297864),isnotnull(PLTYP#297867),isnotnull(MATNR#297868),isnotnull(V
TWEG#297866),isnotnull(VKORG#297865),(KSCHL#297864 = VKP0),(PLTYP#297867 = SL),
(gettimestamp(DATBI#297870, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-06 00:00:00),(gettimestamp(DATAB#297871, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-07 00:00:00),VTWEG#297866 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#297877)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#16075),MATL_CAT#16115 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298115),isnotnull(PLTYP#298118),isnotnull(MATNR#298119),isnotnull(V
TWEG#298117),isnotnull(VKORG#298116),(KSCHL#298115 = VKP0),(PLTYP#298118 = OR),
(gettimestamp(DATBI#298121, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-07 00:00:00),(gettimestamp(DATAB#298122, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-08 00:00:00),VTWEG#298117 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298128)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298195),isnotnull(PLTYP#298198),isnotnull(MATNR#298199),isnotnull(V
TWEG#298197),isnotnull(VKORG#298196),(KSCHL#298195 = VKP0),(PLTYP#298198 = CR),
(gettimestamp(DATBI#298201, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-07 00:00:00),(gettimestamp(DATAB#298202, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-08 00:00:00),VTWEG#298197 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298208)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298286),isnotnull(PLTYP#298289),isnotnull(MATNR#298290),isnotnull(V
TWEG#298288),isnotnull(VKORG#298287),(KSCHL#298286 = VKP0),(PLTYP#298289 = SL),
(gettimestamp(DATBI#298292, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-07 00:00:00),(gettimestamp(DATAB#298293, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-08 00:00:00),VTWEG#298288 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298299)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#20234),MATL_CAT#20274 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298537),isnotnull(PLTYP#298540),isnotnull(MATNR#298541),isnotnull(V
TWEG#298539),isnotnull(VKORG#298538),(KSCHL#298537 = VKP0),(PLTYP#298540 = OR),
(gettimestamp(DATBI#298543, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-08 00:00:00),(gettimestamp(DATAB#298544, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-09 00:00:00),VTWEG#298539 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298550)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298617),isnotnull(PLTYP#298620),isnotnull(MATNR#298621),isnotnull(V
TWEG#298619),isnotnull(VKORG#298618),(KSCHL#298617 = VKP0),(PLTYP#298620 = CR),
(gettimestamp(DATBI#298623, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-08 00:00:00),(gettimestamp(DATAB#298624, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-09 00:00:00),VTWEG#298619 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298630)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298708),isnotnull(PLTYP#298711),isnotnull(MATNR#298712),isnotnull(V
TWEG#298710),isnotnull(VKORG#298709),(KSCHL#298708 = VKP0),(PLTYP#298711 = SL),
(gettimestamp(DATBI#298714, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-08 00:00:00),(gettimestamp(DATAB#298715, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-09 00:00:00),VTWEG#298710 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298721)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#24815),MATL_CAT#24855 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#298959),isnotnull(PLTYP#298962),isnotnull(MATNR#298963),isnotnull(V
TWEG#298961),isnotnull(VKORG#298960),(KSCHL#298959 = VKP0),(PLTYP#298962 = OR),
(gettimestamp(DATBI#298965, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-09 00:00:00),(gettimestamp(DATAB#298966, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-10 00:00:00),VTWEG#298961 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#298972)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299039),isnotnull(PLTYP#299042),isnotnull(MATNR#299043),isnotnull(V
TWEG#299041),isnotnull(VKORG#299040),(KSCHL#299039 = VKP0),(PLTYP#299042 = CR),
(gettimestamp(DATBI#299045, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-09 00:00:00),(gettimestamp(DATAB#299046, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-10 00:00:00),VTWEG#299041 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299052)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299130),isnotnull(PLTYP#299133),isnotnull(MATNR#299134),isnotnull(V
TWEG#299132),isnotnull(VKORG#299131),(KSCHL#299130 = VKP0),(PLTYP#299133 = SL),
(gettimestamp(DATBI#299136, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-09 00:00:00),(gettimestamp(DATAB#299137, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-10 00:00:00),VTWEG#299132 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299143)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#29818),MATL_CAT#29858 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299381),isnotnull(PLTYP#299384),isnotnull(MATNR#299385),isnotnull(V
TWEG#299383),isnotnull(VKORG#299382),(KSCHL#299381 = VKP0),(PLTYP#299384 = OR),
(gettimestamp(DATBI#299387, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-10 00:00:00),(gettimestamp(DATAB#299388, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-11 00:00:00),VTWEG#299383 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299394)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299461),isnotnull(PLTYP#299464),isnotnull(MATNR#299465),isnotnull(V
TWEG#299463),isnotnull(VKORG#299462),(KSCHL#299461 = VKP0),(PLTYP#299464 = CR),
(gettimestamp(DATBI#299467, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-10 00:00:00),(gettimestamp(DATAB#299468, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-11 00:00:00),VTWEG#299463 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299474)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299552),isnotnull(PLTYP#299555),isnotnull(MATNR#299556),isnotnull(V
TWEG#299554),isnotnull(VKORG#299553),(KSCHL#299552 = VKP0),(PLTYP#299555 = SL),
(gettimestamp(DATBI#299558, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-10 00:00:00),(gettimestamp(DATAB#299559, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-11 00:00:00),VTWEG#299554 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299565)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#35243),MATL_CAT#35283 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299803),isnotnull(PLTYP#299806),isnotnull(MATNR#299807),isnotnull(V
TWEG#299805),isnotnull(VKORG#299804),(KSCHL#299803 = VKP0),(PLTYP#299806 = OR),
(gettimestamp(DATBI#299809, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-11 00:00:00),(gettimestamp(DATAB#299810, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-12 00:00:00),VTWEG#299805 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299816)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299883),isnotnull(PLTYP#299886),isnotnull(MATNR#299887),isnotnull(V
TWEG#299885),isnotnull(VKORG#299884),(KSCHL#299883 = VKP0),(PLTYP#299886 = CR),
(gettimestamp(DATBI#299889, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-11 00:00:00),(gettimestamp(DATAB#299890, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-12 00:00:00),VTWEG#299885 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299896)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#299974),isnotnull(PLTYP#299977),isnotnull(MATNR#299978),isnotnull(V
TWEG#299976),isnotnull(VKORG#299975),(KSCHL#299974 = VKP0),(PLTYP#299977 = SL),
(gettimestamp(DATBI#299980, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-11 00:00:00),(gettimestamp(DATAB#299981, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-12 00:00:00),VTWEG#299976 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#299987)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#41090),MATL_CAT#41130 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300225),isnotnull(PLTYP#300228),isnotnull(MATNR#300229),isnotnull(V
TWEG#300227),isnotnull(VKORG#300226),(KSCHL#300225 = VKP0),(PLTYP#300228 = OR),
(gettimestamp(DATBI#300231, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-12 00:00:00),(gettimestamp(DATAB#300232, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-13 00:00:00),VTWEG#300227 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300238)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300305),isnotnull(PLTYP#300308),isnotnull(MATNR#300309),isnotnull(V
TWEG#300307),isnotnull(VKORG#300306),(KSCHL#300305 = VKP0),(PLTYP#300308 = CR),
(gettimestamp(DATBI#300311, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-12 00:00:00),(gettimestamp(DATAB#300312, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-13 00:00:00),VTWEG#300307 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300318)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300396),isnotnull(PLTYP#300399),isnotnull(MATNR#300400),isnotnull(V
TWEG#300398),isnotnull(VKORG#300397),(KSCHL#300396 = VKP0),(PLTYP#300399 = SL),
(gettimestamp(DATBI#300402, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-12 00:00:00),(gettimestamp(DATAB#300403, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-13 00:00:00),VTWEG#300398 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300409)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#47359),MATL_CAT#47399 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300647),isnotnull(PLTYP#300650),isnotnull(MATNR#300651),isnotnull(V
TWEG#300649),isnotnull(VKORG#300648),(KSCHL#300647 = VKP0),(PLTYP#300650 = OR),
(gettimestamp(DATBI#300653, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-13 00:00:00),(gettimestamp(DATAB#300654, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-14 00:00:00),VTWEG#300649 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300660)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300727),isnotnull(PLTYP#300730),isnotnull(MATNR#300731),isnotnull(V
TWEG#300729),isnotnull(VKORG#300728),(KSCHL#300727 = VKP0),(PLTYP#300730 = CR),
(gettimestamp(DATBI#300733, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-13 00:00:00),(gettimestamp(DATAB#300734, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-14 00:00:00),VTWEG#300729 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300740)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#300818),isnotnull(PLTYP#300821),isnotnull(MATNR#300822),isnotnull(V
TWEG#300820),isnotnull(VKORG#300819),(KSCHL#300818 = VKP0),(PLTYP#300821 = SL),
(gettimestamp(DATBI#300824, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-13 00:00:00),(gettimestamp(DATAB#300825, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-14 00:00:00),VTWEG#300820 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#300831)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#54050),MATL_CAT#54090 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301069),isnotnull(PLTYP#301072),isnotnull(MATNR#301073),isnotnull(V
TWEG#301071),isnotnull(VKORG#301070),(KSCHL#301069 = VKP0),(PLTYP#301072 = OR),
(gettimestamp(DATBI#301075, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-14 00:00:00),(gettimestamp(DATAB#301076, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-15 00:00:00),VTWEG#301071 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301082)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301149),isnotnull(PLTYP#301152),isnotnull(MATNR#301153),isnotnull(V
TWEG#301151),isnotnull(VKORG#301150),(KSCHL#301149 = VKP0),(PLTYP#301152 = CR),
(gettimestamp(DATBI#301155, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-14 00:00:00),(gettimestamp(DATAB#301156, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-15 00:00:00),VTWEG#301151 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301162)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301240),isnotnull(PLTYP#301243),isnotnull(MATNR#301244),isnotnull(V
TWEG#301242),isnotnull(VKORG#301241),(KSCHL#301240 = VKP0),(PLTYP#301243 = SL),
(gettimestamp(DATBI#301246, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-14 00:00:00),(gettimestamp(DATAB#301247, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-15 00:00:00),VTWEG#301242 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301253)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#61163),MATL_CAT#61203 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301491),isnotnull(PLTYP#301494),isnotnull(MATNR#301495),isnotnull(V
TWEG#301493),isnotnull(VKORG#301492),(KSCHL#301491 = VKP0),(PLTYP#301494 = OR),
(gettimestamp(DATBI#301497, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-15 00:00:00),(gettimestamp(DATAB#301498, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-16 00:00:00),VTWEG#301493 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301504)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301571),isnotnull(PLTYP#301574),isnotnull(MATNR#301575),isnotnull(V
TWEG#301573),isnotnull(VKORG#301572),(KSCHL#301571 = VKP0),(PLTYP#301574 = CR),
(gettimestamp(DATBI#301577, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-15 00:00:00),(gettimestamp(DATAB#301578, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-16 00:00:00),VTWEG#301573 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301584)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301662),isnotnull(PLTYP#301665),isnotnull(MATNR#301666),isnotnull(V
TWEG#301664),isnotnull(VKORG#301663),(KSCHL#301662 = VKP0),(PLTYP#301665 = SL),
(gettimestamp(DATBI#301668, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-15 00:00:00),(gettimestamp(DATAB#301669, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-16 00:00:00),VTWEG#301664 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301675)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#68698),MATL_CAT#68738 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301913),isnotnull(PLTYP#301916),isnotnull(MATNR#301917),isnotnull(V
TWEG#301915),isnotnull(VKORG#301914),(KSCHL#301913 = VKP0),(PLTYP#301916 = OR),
(gettimestamp(DATBI#301919, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-16 00:00:00),(gettimestamp(DATAB#301920, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-17 00:00:00),VTWEG#301915 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#301926)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#301993),isnotnull(PLTYP#301996),isnotnull(MATNR#301997),isnotnull(V
TWEG#301995),isnotnull(VKORG#301994),(KSCHL#301993 = VKP0),(PLTYP#301996 = CR),
(gettimestamp(DATBI#301999, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-16 00:00:00),(gettimestamp(DATAB#302000, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-17 00:00:00),VTWEG#301995 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302006)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302084),isnotnull(PLTYP#302087),isnotnull(MATNR#302088),isnotnull(V
TWEG#302086),isnotnull(VKORG#302085),(KSCHL#302084 = VKP0),(PLTYP#302087 = SL),
(gettimestamp(DATBI#302090, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-16 00:00:00),(gettimestamp(DATAB#302091, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-17 00:00:00),VTWEG#302086 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302097)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#76655),MATL_CAT#76695 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302335),isnotnull(PLTYP#302338),isnotnull(MATNR#302339),isnotnull(V
TWEG#302337),isnotnull(VKORG#302336),(KSCHL#302335 = VKP0),(PLTYP#302338 = OR),
(gettimestamp(DATBI#302341, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-17 00:00:00),(gettimestamp(DATAB#302342, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-18 00:00:00),VTWEG#302337 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302348)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302415),isnotnull(PLTYP#302418),isnotnull(MATNR#302419),isnotnull(V
TWEG#302417),isnotnull(VKORG#302416),(KSCHL#302415 = VKP0),(PLTYP#302418 = CR),
(gettimestamp(DATBI#302421, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-17 00:00:00),(gettimestamp(DATAB#302422, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-18 00:00:00),VTWEG#302417 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302428)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302506),isnotnull(PLTYP#302509),isnotnull(MATNR#302510),isnotnull(V
TWEG#302508),isnotnull(VKORG#302507),(KSCHL#302506 = VKP0),(PLTYP#302509 = SL),
(gettimestamp(DATBI#302512, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-17 00:00:00),(gettimestamp(DATAB#302513, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-18 00:00:00),VTWEG#302508 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302519)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#85034),MATL_CAT#85074 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302757),isnotnull(PLTYP#302760),isnotnull(MATNR#302761),isnotnull(V
TWEG#302759),isnotnull(VKORG#302758),(KSCHL#302757 = VKP0),(PLTYP#302760 = OR),
(gettimestamp(DATBI#302763, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-18 00:00:00),(gettimestamp(DATAB#302764, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-19 00:00:00),VTWEG#302759 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302770)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302837),isnotnull(PLTYP#302840),isnotnull(MATNR#302841),isnotnull(V
TWEG#302839),isnotnull(VKORG#302838),(KSCHL#302837 = VKP0),(PLTYP#302840 = CR),
(gettimestamp(DATBI#302843, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-18 00:00:00),(gettimestamp(DATAB#302844, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-19 00:00:00),VTWEG#302839 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302850)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#302928),isnotnull(PLTYP#302931),isnotnull(MATNR#302932),isnotnull(V
TWEG#302930),isnotnull(VKORG#302929),(KSCHL#302928 = VKP0),(PLTYP#302931 = SL),
(gettimestamp(DATBI#302934, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-18 00:00:00),(gettimestamp(DATAB#302935, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-19 00:00:00),VTWEG#302930 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#302941)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#93835),MATL_CAT#93875 IN (00,02)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303179),isnotnull(PLTYP#303182),isnotnull(MATNR#303183),isnotnull(V
TWEG#303181),isnotnull(VKORG#303180),(KSCHL#303179 = VKP0),(PLTYP#303182 = OR),
(gettimestamp(DATBI#303185, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-19 00:00:00),(gettimestamp(DATAB#303186, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-20 00:00:00),VTWEG#303181 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303192)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303259),isnotnull(PLTYP#303262),isnotnull(MATNR#303263),isnotnull(V
TWEG#303261),isnotnull(VKORG#303260),(KSCHL#303259 = VKP0),(PLTYP#303262 = CR),
(gettimestamp(DATBI#303265, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-19 00:00:00),(gettimestamp(DATAB#303266, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-20 00:00:00),VTWEG#303261 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303272)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303350),isnotnull(PLTYP#303353),isnotnull(MATNR#303354),isnotnull(V
TWEG#303352),isnotnull(VKORG#303351),(KSCHL#303350 = VKP0),(PLTYP#303353 = SL),
(gettimestamp(DATBI#303356, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-19 00:00:00),(gettimestamp(DATAB#303357, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-20 00:00:00),VTWEG#303352 IN (20,25)
25/02/06 06:07:45 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303363)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#103058),MATL_CAT#103098 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303601),isnotnull(PLTYP#303604),isnotnull(MATNR#303605),isnotnull(V
TWEG#303603),isnotnull(VKORG#303602),(KSCHL#303601 = VKP0),(PLTYP#303604 = OR),
(gettimestamp(DATBI#303607, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-20 00:00:00),(gettimestamp(DATAB#303608, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-21 00:00:00),VTWEG#303603 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303614)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303681),isnotnull(PLTYP#303684),isnotnull(MATNR#303685),isnotnull(V
TWEG#303683),isnotnull(VKORG#303682),(KSCHL#303681 = VKP0),(PLTYP#303684 = CR),
(gettimestamp(DATBI#303687, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-20 00:00:00),(gettimestamp(DATAB#303688, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-21 00:00:00),VTWEG#303683 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303694)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#303772),isnotnull(PLTYP#303775),isnotnull(MATNR#303776),isnotnull(V
TWEG#303774),isnotnull(VKORG#303773),(KSCHL#303772 = VKP0),(PLTYP#303775 = SL),
(gettimestamp(DATBI#303778, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-20 00:00:00),(gettimestamp(DATAB#303779, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-21 00:00:00),VTWEG#303774 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#303785)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#112703),MATL_CAT#112743 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304023),isnotnull(PLTYP#304026),isnotnull(MATNR#304027),isnotnull(V
TWEG#304025),isnotnull(VKORG#304024),(KSCHL#304023 = VKP0),(PLTYP#304026 = OR),
(gettimestamp(DATBI#304029, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-21 00:00:00),(gettimestamp(DATAB#304030, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-22 00:00:00),VTWEG#304025 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304036)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304103),isnotnull(PLTYP#304106),isnotnull(MATNR#304107),isnotnull(V
TWEG#304105),isnotnull(VKORG#304104),(KSCHL#304103 = VKP0),(PLTYP#304106 = CR),
(gettimestamp(DATBI#304109, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-21 00:00:00),(gettimestamp(DATAB#304110, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-22 00:00:00),VTWEG#304105 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304116)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304194),isnotnull(PLTYP#304197),isnotnull(MATNR#304198),isnotnull(V
TWEG#304196),isnotnull(VKORG#304195),(KSCHL#304194 = VKP0),(PLTYP#304197 = SL),
(gettimestamp(DATBI#304200, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-21 00:00:00),(gettimestamp(DATAB#304201, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-22 00:00:00),VTWEG#304196 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304207)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#122770),MATL_CAT#122810 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304445),isnotnull(PLTYP#304448),isnotnull(MATNR#304449),isnotnull(V
TWEG#304447),isnotnull(VKORG#304446),(KSCHL#304445 = VKP0),(PLTYP#304448 = OR),
(gettimestamp(DATBI#304451, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-22 00:00:00),(gettimestamp(DATAB#304452, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-23 00:00:00),VTWEG#304447 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304458)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304525),isnotnull(PLTYP#304528),isnotnull(MATNR#304529),isnotnull(V
TWEG#304527),isnotnull(VKORG#304526),(KSCHL#304525 = VKP0),(PLTYP#304528 = CR),
(gettimestamp(DATBI#304531, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-22 00:00:00),(gettimestamp(DATAB#304532, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-23 00:00:00),VTWEG#304527 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304538)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304616),isnotnull(PLTYP#304619),isnotnull(MATNR#304620),isnotnull(V
TWEG#304618),isnotnull(VKORG#304617),(KSCHL#304616 = VKP0),(PLTYP#304619 = SL),
(gettimestamp(DATBI#304622, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-22 00:00:00),(gettimestamp(DATAB#304623, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-23 00:00:00),VTWEG#304618 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304629)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#133259),MATL_CAT#133299 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304867),isnotnull(PLTYP#304870),isnotnull(MATNR#304871),isnotnull(V
TWEG#304869),isnotnull(VKORG#304868),(KSCHL#304867 = VKP0),(PLTYP#304870 = OR),
(gettimestamp(DATBI#304873, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-23 00:00:00),(gettimestamp(DATAB#304874, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-24 00:00:00),VTWEG#304869 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304880)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#304947),isnotnull(PLTYP#304950),isnotnull(MATNR#304951),isnotnull(V
TWEG#304949),isnotnull(VKORG#304948),(KSCHL#304947 = VKP0),(PLTYP#304950 = CR),
(gettimestamp(DATBI#304953, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-23 00:00:00),(gettimestamp(DATAB#304954, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-24 00:00:00),VTWEG#304949 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#304960)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305038),isnotnull(PLTYP#305041),isnotnull(MATNR#305042),isnotnull(V
TWEG#305040),isnotnull(VKORG#305039),(KSCHL#305038 = VKP0),(PLTYP#305041 = SL),
(gettimestamp(DATBI#305044, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-23 00:00:00),(gettimestamp(DATAB#305045, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-24 00:00:00),VTWEG#305040 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305051)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#144170),MATL_CAT#144210 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305289),isnotnull(PLTYP#305292),isnotnull(MATNR#305293),isnotnull(V
TWEG#305291),isnotnull(VKORG#305290),(KSCHL#305289 = VKP0),(PLTYP#305292 = OR),
(gettimestamp(DATBI#305295, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-24 00:00:00),(gettimestamp(DATAB#305296, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-25 00:00:00),VTWEG#305291 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305302)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305369),isnotnull(PLTYP#305372),isnotnull(MATNR#305373),isnotnull(V
TWEG#305371),isnotnull(VKORG#305370),(KSCHL#305369 = VKP0),(PLTYP#305372 = CR),
(gettimestamp(DATBI#305375, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-24 00:00:00),(gettimestamp(DATAB#305376, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-25 00:00:00),VTWEG#305371 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305382)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305460),isnotnull(PLTYP#305463),isnotnull(MATNR#305464),isnotnull(V
TWEG#305462),isnotnull(VKORG#305461),(KSCHL#305460 = VKP0),(PLTYP#305463 = SL),
(gettimestamp(DATBI#305466, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-24 00:00:00),(gettimestamp(DATAB#305467, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-25 00:00:00),VTWEG#305462 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305473)
25/02/06 06:07:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#155503),MATL_CAT#155543 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305711),isnotnull(PLTYP#305714),isnotnull(MATNR#305715),isnotnull(V
TWEG#305713),isnotnull(VKORG#305712),(KSCHL#305711 = VKP0),(PLTYP#305714 = OR),
(gettimestamp(DATBI#305717, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-25 00:00:00),(gettimestamp(DATAB#305718, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-26 00:00:00),VTWEG#305713 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305724)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305791),isnotnull(PLTYP#305794),isnotnull(MATNR#305795),isnotnull(V
TWEG#305793),isnotnull(VKORG#305792),(KSCHL#305791 = VKP0),(PLTYP#305794 = CR),
(gettimestamp(DATBI#305797, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-25 00:00:00),(gettimestamp(DATAB#305798, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-26 00:00:00),VTWEG#305793 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305804)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#305882),isnotnull(PLTYP#305885),isnotnull(MATNR#305886),isnotnull(V
TWEG#305884),isnotnull(VKORG#305883),(KSCHL#305882 = VKP0),(PLTYP#305885 = SL),
(gettimestamp(DATBI#305888, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-25 00:00:00),(gettimestamp(DATAB#305889, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-26 00:00:00),VTWEG#305884 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#305895)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#167258),MATL_CAT#167298 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306133),isnotnull(PLTYP#306136),isnotnull(MATNR#306137),isnotnull(V
TWEG#306135),isnotnull(VKORG#306134),(KSCHL#306133 = VKP0),(PLTYP#306136 = OR),
(gettimestamp(DATBI#306139, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-26 00:00:00),(gettimestamp(DATAB#306140, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-27 00:00:00),VTWEG#306135 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306146)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306213),isnotnull(PLTYP#306216),isnotnull(MATNR#306217),isnotnull(V
TWEG#306215),isnotnull(VKORG#306214),(KSCHL#306213 = VKP0),(PLTYP#306216 = CR),
(gettimestamp(DATBI#306219, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-26 00:00:00),(gettimestamp(DATAB#306220, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-27 00:00:00),VTWEG#306215 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306226)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306304),isnotnull(PLTYP#306307),isnotnull(MATNR#306308),isnotnull(V
TWEG#306306),isnotnull(VKORG#306305),(KSCHL#306304 = VKP0),(PLTYP#306307 = SL),
(gettimestamp(DATBI#306310, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-26 00:00:00),(gettimestamp(DATAB#306311, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-27 00:00:00),VTWEG#306306 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306317)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#179435),MATL_CAT#179475 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306555),isnotnull(PLTYP#306558),isnotnull(MATNR#306559),isnotnull(V
TWEG#306557),isnotnull(VKORG#306556),(KSCHL#306555 = VKP0),(PLTYP#306558 = OR),
(gettimestamp(DATBI#306561, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-27 00:00:00),(gettimestamp(DATAB#306562, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-28 00:00:00),VTWEG#306557 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306568)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306635),isnotnull(PLTYP#306638),isnotnull(MATNR#306639),isnotnull(V
TWEG#306637),isnotnull(VKORG#306636),(KSCHL#306635 = VKP0),(PLTYP#306638 = CR),
(gettimestamp(DATBI#306641, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-27 00:00:00),(gettimestamp(DATAB#306642, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-28 00:00:00),VTWEG#306637 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306648)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306726),isnotnull(PLTYP#306729),isnotnull(MATNR#306730),isnotnull(V
TWEG#306728),isnotnull(VKORG#306727),(KSCHL#306726 = VKP0),(PLTYP#306729 = SL),
(gettimestamp(DATBI#306732, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-27 00:00:00),(gettimestamp(DATAB#306733, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-28 00:00:00),VTWEG#306728 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306739)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#192034),MATL_CAT#192074 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#306977),isnotnull(PLTYP#306980),isnotnull(MATNR#306981),isnotnull(V
TWEG#306979),isnotnull(VKORG#306978),(KSCHL#306977 = VKP0),(PLTYP#306980 = OR),
(gettimestamp(DATBI#306983, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-28 00:00:00),(gettimestamp(DATAB#306984, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-29 00:00:00),VTWEG#306979 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#306990)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307057),isnotnull(PLTYP#307060),isnotnull(MATNR#307061),isnotnull(V
TWEG#307059),isnotnull(VKORG#307058),(KSCHL#307057 = VKP0),(PLTYP#307060 = CR),
(gettimestamp(DATBI#307063, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-28 00:00:00),(gettimestamp(DATAB#307064, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-29 00:00:00),VTWEG#307059 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307070)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307148),isnotnull(PLTYP#307151),isnotnull(MATNR#307152),isnotnull(V
TWEG#307150),isnotnull(VKORG#307149),(KSCHL#307148 = VKP0),(PLTYP#307151 = SL),
(gettimestamp(DATBI#307154, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-28 00:00:00),(gettimestamp(DATAB#307155, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-29 00:00:00),VTWEG#307150 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307161)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#205055),MATL_CAT#205095 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307399),isnotnull(PLTYP#307402),isnotnull(MATNR#307403),isnotnull(V
TWEG#307401),isnotnull(VKORG#307400),(KSCHL#307399 = VKP0),(PLTYP#307402 = OR),
(gettimestamp(DATBI#307405, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-29 00:00:00),(gettimestamp(DATAB#307406, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-30 00:00:00),VTWEG#307401 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307412)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307479),isnotnull(PLTYP#307482),isnotnull(MATNR#307483),isnotnull(V
TWEG#307481),isnotnull(VKORG#307480),(KSCHL#307479 = VKP0),(PLTYP#307482 = CR),
(gettimestamp(DATBI#307485, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-29 00:00:00),(gettimestamp(DATAB#307486, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-30 00:00:00),VTWEG#307481 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307492)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307570),isnotnull(PLTYP#307573),isnotnull(MATNR#307574),isnotnull(V
TWEG#307572),isnotnull(VKORG#307571),(KSCHL#307570 = VKP0),(PLTYP#307573 = SL),
(gettimestamp(DATBI#307576, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-29 00:00:00),(gettimestamp(DATAB#307577, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-11-30 00:00:00),VTWEG#307572 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307583)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(MATERIAL),In(MATL_CAT, [00,02])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(MATERIAL#218498),MATL_CAT#218538 IN (00,02)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,OR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307821),isnotnull(PLTYP#307824),isnotnull(MATNR#307825),isnotnull(V
TWEG#307823),isnotnull(VKORG#307822),(KSCHL#307821 = VKP0),(PLTYP#307824 = OR),
(gettimestamp(DATBI#307827, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-30 00:00:00),(gettimestamp(DATAB#307828, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-12-01 00:00:00),VTWEG#307823 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307834)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,CR),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307901),isnotnull(PLTYP#307904),isnotnull(MATNR#307905),isnotnull(V
TWEG#307903),isnotnull(VKORG#307902),(KSCHL#307901 = VKP0),(PLTYP#307904 = CR),
(gettimestamp(DATBI#307907, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-30 00:00:00),(gettimestamp(DATAB#307908, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-12-01 00:00:00),VTWEG#307903 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#307914)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(KSCHL),IsNotNull(PLTYP),IsNotNull(MATNR),IsNotNull(VTWEG),IsNotNull(VKORG
),EqualTo(KSCHL,VKP0),EqualTo(PLTYP,SL),In(VTWEG, [20,25])
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KSCHL#307992),isnotnull(PLTYP#307995),isnotnull(MATNR#307996),isnotnull(V
TWEG#307994),isnotnull(VKORG#307993),(KSCHL#307992 = VKP0),(PLTYP#307995 = SL),
(gettimestamp(DATBI#307998, yyyymmdd, TimestampType, Some(Etc/UTC), false) >= 2022-
11-30 00:00:00),(gettimestamp(DATAB#307999, yyyymmdd, TimestampType, Some(Etc/UTC),
false) < 2022-12-01 00:00:00),VTWEG#307994 IN (20,25)
25/02/06 06:07:46 INFO FileSourceStrategy: Pushed Filters: IsNotNull(KNUMH)
25/02/06 06:07:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(KNUMH#308005)
25/02/06 06:07:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:49 WARN SparkStringUtils: Truncated the string representation of a
plan since it was too large. This behavior can be adjusted by setting
'spark.sql.debug.maxToStringFields'.
25/02/06 06:07:50 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2104081) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionStart["sparkPlanInfo"]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.$colon$colon[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[6]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["children"]-
>scala.collection.immutable.Vector[0]-
>org.apache.spark.sql.execution.SparkPlanInfo["metadata"]-
>scala.collection.convert.Wrappers$MapWrapper["ReadSchema"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerial
izer.java:910)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSe
rializer.java:762)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:7
20)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:3
5)
at
com.fasterxml.jackson.databind.ser.std.StdDelegatingSerializer.serialize(StdDelegat
ingSerializer.java:168)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.module.scala.ser.ScalaIterableSerializer.serializeContents(Sc
alaIterableSerializer.scala:62)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents(Iterabl
eSerializerModule.scala:25)
at
com.fasterxml.jackson.module.scala.ser.IterableSerializer.serializeContents$
(IterableSerializerModule.scala:24)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.module.scala.ser.ResolvedIterableSerializer.serializeContents
(IterableSerializerModule.scala:37)
at
com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySeria
lizerBase.java:255)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178
)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2104081)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:521)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerial
izer.java:905)
... 250 more
25/02/06 06:07:52 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:52 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099066) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099066)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 42344046, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 50786778, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 20 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 1
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 4 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 4
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 4
(MapPartitionsRDD[20] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 111 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 16
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 5 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 5
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 5
(MapPartitionsRDD[111] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 126 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 19
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 6 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 6
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 6
(MapPartitionsRDD[126] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 119 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 18
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 7 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 7
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 7
(MapPartitionsRDD[119] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 102 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 14
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 8 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 8
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 8
(MapPartitionsRDD[102] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 134217728 bytes, max partition size: 78129528, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 147 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 21
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 9 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 9
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 9
(MapPartitionsRDD[147] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 110 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 15
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 10 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 10
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 10
(MapPartitionsRDD[110] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 27 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 3
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 3 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 11
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 11
(MapPartitionsRDD[27] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 21 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 2
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 11 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 12
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 12
(MapPartitionsRDD[21] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 153 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 24
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 12 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 13
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 13
(MapPartitionsRDD[153] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 132 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 20
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 13 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 14
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 14
(MapPartitionsRDD[132] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 209 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 37
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 15 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 15
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 15
(MapPartitionsRDD[209] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 198 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 32
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 17 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 16
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 16
(MapPartitionsRDD[198] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 162 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 25
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 18 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 17
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 17
(MapPartitionsRDD[162] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 184 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 28
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 19 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 18
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 18
(MapPartitionsRDD[184] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 207 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 35
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 20 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 19
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 19
(MapPartitionsRDD[207] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 200 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 36
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 14 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 20
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 20
(MapPartitionsRDD[200] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 202 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 33
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 16 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 21
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 21
(MapPartitionsRDD[202] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 212 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 38
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 21 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 22
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 22
(MapPartitionsRDD[212] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 226 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 41
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 22 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 23
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 23
(MapPartitionsRDD[226] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 232 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 42
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 23 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 24
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 24
(MapPartitionsRDD[232] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 236 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 43
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 24 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 25
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 25
(MapPartitionsRDD[236] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 241 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 44
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 25 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 26
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 26
(MapPartitionsRDD[241] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 167 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 26
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 26 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 27
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 27
(MapPartitionsRDD[167] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 105 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 13
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 28 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 28
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 28
(MapPartitionsRDD[105] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 78 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 6
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 27 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 29
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 29
(MapPartitionsRDD[78] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 249 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 45
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 29 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 30
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 30
(MapPartitionsRDD[249] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 253 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 46
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 30 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 31
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 31
(MapPartitionsRDD[253] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 100 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 12
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 31 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 32
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:52 INFO DAGScheduler: Submitting ShuffleMapStage 32
(MapPartitionsRDD[100] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:52 INFO DAGScheduler: Registering RDD 257 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 47
25/02/06 06:07:52 INFO DAGScheduler: Got map stage job 32 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:52 INFO DAGScheduler: Final stage: ShuffleMapStage 33
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:52 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 33
(MapPartitionsRDD[257] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 108 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 17
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 33 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 34
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 34
(MapPartitionsRDD[108] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 93 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 10
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 34 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 35
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 35
(MapPartitionsRDD[93] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 55 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 5
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 35 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 36
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 36
(MapPartitionsRDD[55] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 86 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 9
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 36 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 37
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 37
(MapPartitionsRDD[86] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 85 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 8
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 37 (submitMapStage at
PhotonExec.scala:495) with 3 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 38
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 38
(MapPartitionsRDD[85] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 33 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 4
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 38 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 39
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 39
(MapPartitionsRDD[33] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 265 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 49
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 39 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 40
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 40
(MapPartitionsRDD[265] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 273 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 50
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 40 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 41
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 41
(MapPartitionsRDD[273] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 79 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 7
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 41 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 42
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 42
(MapPartitionsRDD[79] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 197 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 30
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 42 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 43
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 43
(MapPartitionsRDD[197] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 150 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 23
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 43 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 44
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 44
(MapPartitionsRDD[150] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 217 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 39
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 45 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 45
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 45
(MapPartitionsRDD[217] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 220 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 40
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 44 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 46
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 46
(MapPartitionsRDD[220] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 290 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 53
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 46 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 47
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 47
(MapPartitionsRDD[290] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 148 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 22
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 48 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 48
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 48
(MapPartitionsRDD[148] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 205 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 34
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 49 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 49
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 49
(MapPartitionsRDD[205] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 94 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 11
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 47 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 50
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 50
(MapPartitionsRDD[94] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 199 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 31
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 50 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 51
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 51
(MapPartitionsRDD[199] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 187 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 29
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 51 (submitMapStage at
PhotonExec.scala:495) with 8 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 52
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 52
(MapPartitionsRDD[187] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 179 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 27
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 52 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 53
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 53
(MapPartitionsRDD[179] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 321 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 58
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 53 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 54
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 54
(MapPartitionsRDD[321] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 338 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 60
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 54 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 55
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 55
(MapPartitionsRDD[338] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 327 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 59
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 55 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 56
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 56
(MapPartitionsRDD[327] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 303 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 56
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 57 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 57
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 57
(MapPartitionsRDD[303] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 291 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 55
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 59 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 58
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 58
(MapPartitionsRDD[291] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 359 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 65
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 61 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 59
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 59
(MapPartitionsRDD[359] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 362 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 66
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 62 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 60
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 60
(MapPartitionsRDD[362] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 320 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 57
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 58 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 61
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 61
(MapPartitionsRDD[320] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 348 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 62
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 63 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 62
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 62
(MapPartitionsRDD[348] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 302 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 61
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 56 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 63
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 63
(MapPartitionsRDD[302] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 371 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 67
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 65 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 64
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 64
(MapPartitionsRDD[371] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 354 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 63
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 64 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 65
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 65
(MapPartitionsRDD[354] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 351 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 64
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 60 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 66
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 66
(MapPartitionsRDD[351] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 373 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 71
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 68 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 67
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 67
(MapPartitionsRDD[373] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 385 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 69
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 69 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 68
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 68
(MapPartitionsRDD[385] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 377 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 68
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 66 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 69
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 69
(MapPartitionsRDD[377] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 388 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 70
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 70 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 70
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 70
(MapPartitionsRDD[388] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 278 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 51
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 71 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 71
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 71
(MapPartitionsRDD[278] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 381 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 72
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 67 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 72
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 72
(MapPartitionsRDD[381] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 397 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 73
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 72 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 73
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 73
(MapPartitionsRDD[397] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 398 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 74
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 73 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 74
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 74
(MapPartitionsRDD[398] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 404 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 76
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 74 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 75
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 75
(MapPartitionsRDD[404] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 411 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 77
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 75 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 76
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 76
(MapPartitionsRDD[411] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 391 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 79
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 77 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 77
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 77
(MapPartitionsRDD[391] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 259 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 48
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 78 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 78
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 78
(MapPartitionsRDD[259] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 284 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 52
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 79 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 79
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 79
(MapPartitionsRDD[284] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 292 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 54
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 80 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 80
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 80
(MapPartitionsRDD[292] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 407 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 78
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 76 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 81
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 81
(MapPartitionsRDD[407] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 430 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 82
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 81 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 82
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 82
(MapPartitionsRDD[430] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 402 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 75
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 82 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 83
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 83
(MapPartitionsRDD[402] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 422 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 80
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 83 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 84
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 84
(MapPartitionsRDD[422] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 424 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 81
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 84 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 85
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 85
(MapPartitionsRDD[424] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 426 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 83
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 85 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 86
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 86
(MapPartitionsRDD[426] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 445 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 84
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 86 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 87
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 87
(MapPartitionsRDD[445] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 447 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 86
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 87 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 88
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 88
(MapPartitionsRDD[447] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 441 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 87
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 89 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 89
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 89
(MapPartitionsRDD[441] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 452 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 88
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 88 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 90
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 90
(MapPartitionsRDD[452] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 455 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 89
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 90 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 91
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 91
(MapPartitionsRDD[455] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 462 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 90
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 91 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 92
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 92
(MapPartitionsRDD[462] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 466 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 91
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 92 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 93
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 93
(MapPartitionsRDD[466] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 448 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 85
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 93 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 94
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 94
(MapPartitionsRDD[448] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Registering RDD 470 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 92
25/02/06 06:07:53 INFO DAGScheduler: Got map stage job 94 (submitMapStage at
PhotonExec.scala:495) with 5 output partitions
25/02/06 06:07:53 INFO DAGScheduler: Final stage: ShuffleMapStage 95
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:07:53 INFO DAGScheduler: Parents of final stage: List()
25/02/06 06:07:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:07:53 INFO DAGScheduler: Submitting ShuffleMapStage 95
(MapPartitionsRDD[470] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 4 (MapPartitionsRDD[20] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 4.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 4.0 using PreferredLocationsV1
25/02/06 06:07:53 WARN FairSchedulableBuilder: A job was submitted with scheduler
pool 5400065285585087590, which has not been configured. This can happen when the
file that pools are read from isn't set, or when that file doesn't contain
5400065285585087590. Created 5400065285585087590 with default configuration
(schedulingMode: FIFO, minShare: 0, weight: 1)
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_4.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 11 (MapPartitionsRDD[27] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 11.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 11.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_11.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 10 (MapPartitionsRDD[110] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 10.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: Starting task 0.0 in stage 11.0 (TID 3)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 1.0 in stage 11.0 (TID 4)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 2.0 in stage 11.0 (TID 5)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 3.0 in stage 11.0 (TID 6)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 4.0 in stage 11.0 (TID 7)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 8)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 9)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: Starting task 2.0 in stage 4.0 (TID 10)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 10.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_10.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 8 (MapPartitionsRDD[102] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 8.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 8.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_8.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 9 (MapPartitionsRDD[147] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 9.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 9.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_9.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 7 (MapPartitionsRDD[119] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 7.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 7.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_7.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 17 (MapPartitionsRDD[162] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 17.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 17.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_17.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 18 (MapPartitionsRDD[184] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 18.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 18.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_18.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 19 (MapPartitionsRDD[207] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 19.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 19.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_19.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 6 (MapPartitionsRDD[126] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 6.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 6.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_6.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 21 (MapPartitionsRDD[202] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 21.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 21.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_21.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 5 (MapPartitionsRDD[111] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 5.0 with 5 tasks resource
profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 5.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_5.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 23 (MapPartitionsRDD[226] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 23.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 23.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_23.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 24 (MapPartitionsRDD[232] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 24.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 24.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_24.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 25 (MapPartitionsRDD[236] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 25.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 25.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_25.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 22 (MapPartitionsRDD[212] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 22.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 22.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_22.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 20 (MapPartitionsRDD[200] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 20.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 20.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_20.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 16 (MapPartitionsRDD[198] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 16.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 16.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_16.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 28 (MapPartitionsRDD[105] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 28.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 28.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_28.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 27 (MapPartitionsRDD[167] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 27.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 27.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_27.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 29 (MapPartitionsRDD[78] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 29.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 29.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_29.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 15 (MapPartitionsRDD[209] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 15.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 15.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_15.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 33 (MapPartitionsRDD[257] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 33.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 33.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_33.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 34 (MapPartitionsRDD[108] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 34.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 34.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_34.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 30 (MapPartitionsRDD[249] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 30.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 30.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_30.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 26 (MapPartitionsRDD[241] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 26.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 26.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_26.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 14 (MapPartitionsRDD[132] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 14.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 14.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_14.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 13 (MapPartitionsRDD[153] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 13.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 13.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_13.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 12 (MapPartitionsRDD[21] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 12.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 12.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_12.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 39 (MapPartitionsRDD[33] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 39.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 39.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_39.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 40 (MapPartitionsRDD[265] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 40.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 40.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_40.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 3 missing tasks from
ShuffleMapStage 38 (MapPartitionsRDD[85] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 38.0 with 3 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 38.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_38.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 37 (MapPartitionsRDD[86] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 37.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 37.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_37.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 36 (MapPartitionsRDD[55] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 36.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 36.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_36.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 35 (MapPartitionsRDD[93] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 35.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 35.0 using PreferredLocationsV1
25/02/06 06:07:53 INFO FairSchedulableBuilder: Added task set TaskSet_35.0 tasks to
pool 5400065285585087590
25/02/06 06:07:53 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 42 (MapPartitionsRDD[79] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:53 INFO TaskSchedulerImpl: Adding task set 42.0 with 5 tasks
resource profile 0
25/02/06 06:07:53 INFO TaskSetManager: TaskSet 42.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_42.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 32 (MapPartitionsRDD[100] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 32.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 32.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_32.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 31 (MapPartitionsRDD[253] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 31.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 31.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_31.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 43 (MapPartitionsRDD[197] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 43.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 43.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_43.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 45 (MapPartitionsRDD[217] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 45.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 45.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_45.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 44 (MapPartitionsRDD[150] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 44.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 44.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_44.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 41 (MapPartitionsRDD[273] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 41.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 41.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_41.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 50 (MapPartitionsRDD[94] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 50.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 50.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_50.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 49 (MapPartitionsRDD[205] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 49.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 49.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_49.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 48 (MapPartitionsRDD[148] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 48.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 48.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_48.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 47 (MapPartitionsRDD[290] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 47.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 47.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_47.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 46 (MapPartitionsRDD[220] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 46.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 46.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_46.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 57 (MapPartitionsRDD[303] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 57.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 57.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_57.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 56 (MapPartitionsRDD[327] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 56.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 56.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_56.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 55 (MapPartitionsRDD[338] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 55.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 55.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_55.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 54 (MapPartitionsRDD[321] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 54.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 54.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_54.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 53 (MapPartitionsRDD[179] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 53.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 53.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_53.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 8 missing tasks from
ShuffleMapStage 52 (MapPartitionsRDD[187] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 52.0 with 8 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 52.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_52.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 51 (MapPartitionsRDD[199] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 51.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 51.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_51.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 58 (MapPartitionsRDD[291] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 58.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 58.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_58.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 59 (MapPartitionsRDD[359] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 59.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 59.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_59.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 60 (MapPartitionsRDD[362] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 60.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 60.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_60.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 61 (MapPartitionsRDD[320] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 61.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 61.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_61.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 62 (MapPartitionsRDD[348] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 62.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 62.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_62.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 63 (MapPartitionsRDD[302] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 63.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 63.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_63.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 64 (MapPartitionsRDD[371] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 64.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 64.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_64.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 65 (MapPartitionsRDD[354] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 65.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 65.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_65.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 66 (MapPartitionsRDD[351] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 66.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 66.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_66.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 67 (MapPartitionsRDD[373] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 67.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 67.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_67.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 68 (MapPartitionsRDD[385] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 68.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 68.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_68.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 69 (MapPartitionsRDD[377] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 69.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 69.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_69.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 70 (MapPartitionsRDD[388] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 70.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 70.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_70.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 71 (MapPartitionsRDD[278] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 71.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 71.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_71.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 72 (MapPartitionsRDD[381] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 72.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 72.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_72.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 73 (MapPartitionsRDD[397] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 73.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 73.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_73.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 74 (MapPartitionsRDD[398] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 74.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 74.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_74.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 75 (MapPartitionsRDD[404] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 75.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 75.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_75.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 76 (MapPartitionsRDD[411] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 76.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 76.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_76.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 77 (MapPartitionsRDD[391] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 77.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 77.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_77.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 78 (MapPartitionsRDD[259] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 78.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 78.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_78.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 79 (MapPartitionsRDD[284] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 79.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 79.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_79.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 80 (MapPartitionsRDD[292] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 80.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 80.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_80.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 81 (MapPartitionsRDD[407] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 81.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 81.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_81.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 82 (MapPartitionsRDD[430] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 82.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 82.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_82.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 83 (MapPartitionsRDD[402] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 83.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 83.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_83.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 84 (MapPartitionsRDD[422] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 84.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 84.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_84.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 85 (MapPartitionsRDD[424] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 85.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 85.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_85.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 86 (MapPartitionsRDD[426] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 86.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 86.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_86.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 87 (MapPartitionsRDD[445] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 87.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 87.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_87.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 88 (MapPartitionsRDD[447] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 88.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 88.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_88.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 89 (MapPartitionsRDD[441] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 89.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 89.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_89.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 90 (MapPartitionsRDD[452] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 90.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 90.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_90.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 91 (MapPartitionsRDD[455] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 91.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 91.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_91.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 92 (MapPartitionsRDD[462] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 92.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 92.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_92.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 93 (MapPartitionsRDD[466] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 93.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 93.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_93.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 94 (MapPartitionsRDD[448] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 94.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 94.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_94.0 tasks to
pool 5400065285585087590
25/02/06 06:07:54 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 95 (MapPartitionsRDD[470] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
25/02/06 06:07:54 INFO TaskSchedulerImpl: Adding task set 95.0 with 5 tasks
resource profile 0
25/02/06 06:07:54 INFO TaskSetManager: TaskSet 95.0 using PreferredLocationsV1
25/02/06 06:07:54 INFO FairSchedulableBuilder: Added task set TaskSet_95.0 tasks to
pool 5400065285585087590
25/02/06 06:07:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:55 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.229.123:38441 (size: 18.4 KiB, free: 18.4 GiB)
25/02/06 06:07:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:07:59 INFO TaskSetManager: Starting task 3.0 in stage 4.0 (TID 11)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:07:59 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 9) in
5285 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:07:59 INFO TaskSetManager: Starting task 4.0 in stage 4.0 (TID 12)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:07:59 INFO TaskSetManager: Finished task 2.0 in stage 4.0 (TID 10) in
5319 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:07:59 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 13)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:07:59 INFO TaskSetManager: Finished task 3.0 in stage 11.0 (TID 6) in
5329 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:07:59 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 14)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:07:59 INFO TaskSetManager: Finished task 1.0 in stage 11.0 (TID 4) in
5615 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:02 INFO TaskSetManager: Starting task 2.0 in stage 5.0 (TID 15)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:02 INFO TaskSetManager: Finished task 3.0 in stage 4.0 (TID 11) in
3722 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:02 INFO TaskSetManager: Starting task 3.0 in stage 5.0 (TID 16)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:02 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 14) in
3467 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:03 INFO TaskSetManager: Starting task 4.0 in stage 5.0 (TID 17)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:03 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 13) in
4132 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:03 INFO TaskSetManager: Finished task 4.0 in stage 4.0 (TID 12) in
4156 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:03 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 18)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:06 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 19)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:06 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 15) in
3330 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:06 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 20)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:06 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 16) in
3284 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:06 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 21)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:06 INFO TaskSetManager: Finished task 4.0 in stage 5.0 (TID 17) in
3425 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:06 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:06 INFO DAGScheduler: ShuffleMapStage 5 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 14.073 s
25/02/06 06:08:06 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:06 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 4,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44,
ShuffleMapStage 23, ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 11,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34,
ShuffleMapStage 13, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 28, ShuffleMapStage 7, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 9, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 10, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 12,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 6, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29, ShuffleMapStage 8)
25/02/06 06:08:06 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:06 INFO DAGScheduler: failed: Set()
25/02/06 06:08:06 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 18) in
3444 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:06 INFO TaskSetManager: Starting task 4.0 in stage 6.0 (TID 22)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:07 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 23)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:07 INFO TaskSetManager: Finished task 0.0 in stage 11.0 (TID 3) in
13430 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:07 INFO TaskSetManager: Starting task 1.0 in stage 7.0 (TID 24)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:07 INFO TaskSetManager: Starting task 2.0 in stage 7.0 (TID 25)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:07 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 8) in
13435 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:07 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:07 INFO TaskSetManager: Finished task 2.0 in stage 11.0 (TID 5) in
13437 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:07 INFO DAGScheduler: ShuffleMapStage 4 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 14.684 s
25/02/06 06:08:07 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:07 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 11, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28,
ShuffleMapStage 7, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 30, ShuffleMapStage 9, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 14,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 18, ShuffleMapStage 10, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 12, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 6,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29,
ShuffleMapStage 8)
25/02/06 06:08:07 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:07 INFO DAGScheduler: failed: Set()
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:07 INFO TaskSetManager: Starting task 3.0 in stage 7.0 (TID 26)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:07 INFO TaskSetManager: Finished task 4.0 in stage 11.0 (TID 7) in
13558 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:07 INFO TaskSchedulerImpl: Removed TaskSet 11.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:07 INFO DAGScheduler: ShuffleMapStage 11 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 14.647 s
25/02/06 06:08:07 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:07 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 7,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 9, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 20, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 14, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18,
ShuffleMapStage 10, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 12, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 6, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29, ShuffleMapStage 8)
25/02/06 06:08:07 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:07 INFO DAGScheduler: failed: Set()
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:07 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:07 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:08 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:08 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:09 INFO TaskSetManager: Starting task 4.0 in stage 7.0 (TID 27)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:09 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 19) in
3490 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:09 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 28)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:09 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 20) in
3590 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:09 INFO TaskSetManager: Starting task 1.0 in stage 8.0 (TID 29)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:09 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 21) in
3148 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:10 INFO TaskSetManager: Finished task 4.0 in stage 6.0 (TID 22) in
3231 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:10 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:10 INFO TaskSetManager: Starting task 2.0 in stage 8.0 (TID 30)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:10 INFO DAGScheduler: ShuffleMapStage 6 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 17.311 s
25/02/06 06:08:10 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:10 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 7,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 9, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 20, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 14, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18,
ShuffleMapStage 10, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 12, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58, ShuffleMapStage 29, ShuffleMapStage 8)
25/02/06 06:08:10 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:10 INFO DAGScheduler: failed: Set()
25/02/06 06:08:10 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:11 INFO TaskSetManager: Starting task 3.0 in stage 8.0 (TID 31)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:11 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 23) in
3980 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:11 INFO TaskSetManager: Starting task 4.0 in stage 8.0 (TID 32)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:11 INFO TaskSetManager: Finished task 1.0 in stage 7.0 (TID 24) in
4031 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:11 INFO TaskSetManager: Starting task 0.0 in stage 9.0 (TID 33)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:11 INFO TaskSetManager: Finished task 3.0 in stage 7.0 (TID 26) in
3941 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:11 INFO TaskSetManager: Starting task 1.0 in stage 9.0 (TID 34)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:11 INFO TaskSetManager: Finished task 2.0 in stage 7.0 (TID 25) in
4144 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:13 INFO TaskSetManager: Starting task 2.0 in stage 9.0 (TID 35)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:13 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 28) in
3346 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:13 INFO TaskSetManager: Starting task 3.0 in stage 9.0 (TID 36)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:13 INFO TaskSetManager: Finished task 1.0 in stage 8.0 (TID 29) in
3362 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:13 INFO TaskSetManager: Starting task 4.0 in stage 9.0 (TID 37)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:13 INFO TaskSetManager: Finished task 4.0 in stage 7.0 (TID 27) in
3754 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:13 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:13 INFO DAGScheduler: ShuffleMapStage 7 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 20.711 s
25/02/06 06:08:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:13 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 9,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 20, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 22, ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 10,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 12, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29, ShuffleMapStage 8)
25/02/06 06:08:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:13 INFO DAGScheduler: failed: Set()
25/02/06 06:08:13 INFO TaskSetManager: Starting task 0.0 in stage 10.0 (TID 38)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:13 INFO TaskSetManager: Finished task 2.0 in stage 8.0 (TID 30) in
3454 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:14 INFO TaskSetManager: Starting task 1.0 in stage 10.0 (TID 39)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:14 INFO TaskSetManager: Finished task 3.0 in stage 8.0 (TID 31) in
3539 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:14 INFO TaskSetManager: Starting task 2.0 in stage 10.0 (TID 40)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:14 INFO TaskSetManager: Finished task 4.0 in stage 8.0 (TID 32) in
3504 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:14 INFO TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:14 INFO DAGScheduler: ShuffleMapStage 8 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 22.080 s
25/02/06 06:08:14 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:14 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 9,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 20, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 22, ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 10,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 12, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29)
25/02/06 06:08:14 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:14 INFO DAGScheduler: failed: Set()
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:14 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:14 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:15 INFO TaskSetManager: Starting task 3.0 in stage 10.0 (TID 41)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:15 INFO TaskSetManager: Finished task 0.0 in stage 9.0 (TID 33) in
3692 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:15 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:15 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:15 INFO TaskSetManager: Starting task 4.0 in stage 10.0 (TID 42)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:15 INFO TaskSetManager: Finished task 1.0 in stage 9.0 (TID 34) in
4024 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:16 INFO TaskSetManager: Starting task 0.0 in stage 12.0 (TID 43)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:16 INFO TaskSetManager: Finished task 2.0 in stage 9.0 (TID 35) in
3698 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:16 INFO TaskSetManager: Starting task 1.0 in stage 12.0 (TID 44)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:16 INFO TaskSetManager: Finished task 3.0 in stage 9.0 (TID 36) in
3677 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:17 INFO TaskSetManager: Starting task 2.0 in stage 12.0 (TID 45)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:17 INFO TaskSetManager: Finished task 4.0 in stage 9.0 (TID 37) in
3623 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:17 INFO TaskSchedulerImpl: Removed TaskSet 9.0, whose tasks have all
completed, from pool 5400065285585087590
25/02/06 06:08:17 INFO DAGScheduler: ShuffleMapStage 9 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 24.295 s
25/02/06 06:08:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:17 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 10, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 12,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:17 INFO DAGScheduler: failed: Set()
25/02/06 06:08:17 INFO TaskSetManager: Starting task 3.0 in stage 12.0 (TID 46)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:17 INFO TaskSetManager: Finished task 0.0 in stage 10.0 (TID 38) in
3685 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:18 INFO TaskSetManager: Starting task 4.0 in stage 12.0 (TID 47)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:18 INFO TaskSetManager: Finished task 1.0 in stage 10.0 (TID 39) in
3877 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:18 INFO TaskSetManager: Starting task 0.0 in stage 13.0 (TID 48)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:18 INFO TaskSetManager: Finished task 3.0 in stage 10.0 (TID 41) in
3791 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:19 INFO TaskSetManager: Starting task 1.0 in stage 13.0 (TID 49)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:19 INFO TaskSetManager: Finished task 2.0 in stage 10.0 (TID 40) in
4237 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:19 INFO TaskSetManager: Starting task 2.0 in stage 13.0 (TID 50)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:19 INFO TaskSetManager: Finished task 4.0 in stage 10.0 (TID 42) in
3879 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:19 INFO TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:19 INFO DAGScheduler: ShuffleMapStage 10 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 26.570 s
25/02/06 06:08:19 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:19 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 12, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:19 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:19 INFO DAGScheduler: failed: Set()
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:19 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:19 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:20 INFO TaskSetManager: Starting task 3.0 in stage 13.0 (TID 51)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:20 INFO TaskSetManager: Finished task 1.0 in stage 12.0 (TID 44) in
3699 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:20 INFO TaskSetManager: Starting task 4.0 in stage 13.0 (TID 52)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:20 INFO TaskSetManager: Finished task 0.0 in stage 12.0 (TID 43) in
3769 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:20 INFO TaskSetManager: Starting task 0.0 in stage 14.0 (TID 53)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:20 INFO TaskSetManager: Finished task 2.0 in stage 12.0 (TID 45) in
3616 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:20 INFO TaskSetManager: Starting task 1.0 in stage 14.0 (TID 54)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:20 INFO TaskSetManager: Finished task 3.0 in stage 12.0 (TID 46) in
3697 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:22 INFO TaskSetManager: Starting task 2.0 in stage 14.0 (TID 55)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:22 INFO TaskSetManager: Finished task 0.0 in stage 13.0 (TID 48) in
3670 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:22 INFO TaskSetManager: Starting task 3.0 in stage 14.0 (TID 56)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:22 INFO TaskSetManager: Finished task 4.0 in stage 12.0 (TID 47) in
3837 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:22 INFO TaskSchedulerImpl: Removed TaskSet 12.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:22 INFO DAGScheduler: ShuffleMapStage 12 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 29.741 s
25/02/06 06:08:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:22 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 13, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 14, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:22 INFO DAGScheduler: failed: Set()
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:22 INFO TaskSetManager: Finished task 1.0 in stage 13.0 (TID 49) in
3590 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:22 INFO TaskSetManager: Starting task 4.0 in stage 14.0 (TID 57)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:22 INFO TaskSetManager: Starting task 0.0 in stage 20.0 (TID 58)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:22 INFO TaskSetManager: Finished task 2.0 in stage 13.0 (TID 50) in
3392 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:22 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:22 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:08:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:08:23 INFO TaskSetManager: Starting task 1.0 in stage 20.0 (TID 59)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:23 INFO TaskSetManager: Finished task 4.0 in stage 13.0 (TID 52) in
3308 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:23 INFO TaskSetManager: Starting task 2.0 in stage 20.0 (TID 60)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:23 INFO TaskSetManager: Finished task 3.0 in stage 13.0 (TID 51) in
3346 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:23 INFO TaskSchedulerImpl: Removed TaskSet 13.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:23 INFO DAGScheduler: ShuffleMapStage 13 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 31.177 s
25/02/06 06:08:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:23 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 14,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 18, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29)
25/02/06 06:08:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:23 INFO DAGScheduler: failed: Set()
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:24 INFO TaskSetManager: Starting task 3.0 in stage 20.0 (TID 61)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:24 INFO TaskSetManager: Finished task 0.0 in stage 14.0 (TID 53) in
3587 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:24 INFO TaskSetManager: Starting task 4.0 in stage 20.0 (TID 62)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:24 INFO TaskSetManager: Finished task 1.0 in stage 14.0 (TID 54) in
3705 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:26 INFO TaskSetManager: Finished task 3.0 in stage 14.0 (TID 56) in
3658 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:26 INFO TaskSetManager: Starting task 0.0 in stage 15.0 (TID 63)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:26 INFO TaskSetManager: Starting task 1.0 in stage 15.0 (TID 64)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:26 INFO TaskSetManager: Finished task 2.0 in stage 14.0 (TID 55) in
3705 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:26 INFO TaskSetManager: Starting task 2.0 in stage 15.0 (TID 65)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:26 INFO TaskSetManager: Starting task 3.0 in stage 15.0 (TID 66)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:26 INFO TaskSetManager: Finished task 0.0 in stage 20.0 (TID 58) in
3777 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:26 INFO TaskSetManager: Finished task 4.0 in stage 14.0 (TID 57) in
3855 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:26 INFO TaskSchedulerImpl: Removed TaskSet 14.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:26 INFO DAGScheduler: ShuffleMapStage 14 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 33.709 s
25/02/06 06:08:26 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:26 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 20, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:26 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:26 INFO DAGScheduler: failed: Set()
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:27 INFO TaskSetManager: Starting task 4.0 in stage 15.0 (TID 67)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:27 INFO TaskSetManager: Finished task 1.0 in stage 20.0 (TID 59) in
3703 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:27 INFO TaskSetManager: Starting task 0.0 in stage 21.0 (TID 68)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:27 INFO TaskSetManager: Finished task 2.0 in stage 20.0 (TID 60) in
3709 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:27 INFO TaskSetManager: Starting task 1.0 in stage 21.0 (TID 69)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:27 INFO TaskSetManager: Finished task 3.0 in stage 20.0 (TID 61) in
3551 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:28 INFO TaskSetManager: Starting task 2.0 in stage 21.0 (TID 70)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:28 INFO TaskSetManager: Finished task 4.0 in stage 20.0 (TID 62) in
4041 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:28 INFO TaskSchedulerImpl: Removed TaskSet 20.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:28 INFO DAGScheduler: ShuffleMapStage 20 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 35.719 s
25/02/06 06:08:28 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:28 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 15, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:28 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:28 INFO DAGScheduler: failed: Set()
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:29 INFO TaskSetManager: Starting task 3.0 in stage 21.0 (TID 71)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:29 INFO TaskSetManager: Finished task 0.0 in stage 15.0 (TID 63) in
3781 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:30 INFO TaskSetManager: Starting task 4.0 in stage 21.0 (TID 72)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:30 INFO TaskSetManager: Finished task 3.0 in stage 15.0 (TID 66) in
3615 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:30 INFO TaskSetManager: Starting task 0.0 in stage 16.0 (TID 73)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:30 INFO TaskSetManager: Finished task 2.0 in stage 15.0 (TID 65) in
3687 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:30 INFO TaskSetManager: Starting task 1.0 in stage 16.0 (TID 74)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:30 INFO TaskSetManager: Finished task 1.0 in stage 15.0 (TID 64) in
3984 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:31 INFO TaskSetManager: Starting task 2.0 in stage 16.0 (TID 75)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:31 INFO TaskSetManager: Finished task 0.0 in stage 21.0 (TID 68) in
3584 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:31 INFO TaskSetManager: Starting task 3.0 in stage 16.0 (TID 76)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:31 INFO TaskSetManager: Finished task 4.0 in stage 15.0 (TID 67) in
3787 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:31 INFO TaskSchedulerImpl: Removed TaskSet 15.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:31 INFO DAGScheduler: ShuffleMapStage 15 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 38.614 s
25/02/06 06:08:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:31 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 21, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38,
ShuffleMapStage 17, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 22, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:31 INFO DAGScheduler: failed: Set()
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:31 INFO TaskSetManager: Starting task 4.0 in stage 16.0 (TID 77)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:31 INFO TaskSetManager: Finished task 1.0 in stage 21.0 (TID 69) in
3740 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:31 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:31 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:32 INFO TaskSetManager: Starting task 0.0 in stage 17.0 (TID 78)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:32 INFO TaskSetManager: Finished task 2.0 in stage 21.0 (TID 70) in
3641 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:33 INFO TaskSetManager: Starting task 1.0 in stage 17.0 (TID 79)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:33 INFO TaskSetManager: Finished task 4.0 in stage 21.0 (TID 72) in
3584 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:33 INFO TaskSetManager: Starting task 2.0 in stage 17.0 (TID 80)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:33 INFO TaskSetManager: Finished task 3.0 in stage 21.0 (TID 71) in
3833 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:33 INFO TaskSchedulerImpl: Removed TaskSet 21.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:33 INFO DAGScheduler: ShuffleMapStage 21 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 40.906 s
25/02/06 06:08:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:33 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 17,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 16, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 18, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29)
25/02/06 06:08:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:33 INFO DAGScheduler: failed: Set()
25/02/06 06:08:33 INFO TaskSetManager: Starting task 3.0 in stage 17.0 (TID 81)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:33 INFO TaskSetManager: Finished task 0.0 in stage 16.0 (TID 73) in
3642 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:33 INFO TaskSetManager: Starting task 4.0 in stage 17.0 (TID 82)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:33 INFO TaskSetManager: Finished task 1.0 in stage 16.0 (TID 74) in
3758 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:34 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:34 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:34 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:34 INFO TaskSetManager: Starting task 0.0 in stage 18.0 (TID 83)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:34 INFO TaskSetManager: Finished task 2.0 in stage 16.0 (TID 75) in
3648 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:34 INFO TaskSetManager: Starting task 1.0 in stage 18.0 (TID 84)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:34 INFO TaskSetManager: Finished task 3.0 in stage 16.0 (TID 76) in
3589 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:35 INFO TaskSetManager: Starting task 2.0 in stage 18.0 (TID 85)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:35 INFO TaskSetManager: Finished task 4.0 in stage 16.0 (TID 77) in
3556 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:35 INFO TaskSchedulerImpl: Removed TaskSet 16.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:35 INFO DAGScheduler: ShuffleMapStage 16 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 42.292 s
25/02/06 06:08:35 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:35 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 17,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:35 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:35 INFO DAGScheduler: failed: Set()
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:35 INFO TaskSetManager: Starting task 3.0 in stage 18.0 (TID 86)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:35 INFO TaskSetManager: Finished task 0.0 in stage 17.0 (TID 78) in
3564 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:37 INFO TaskSetManager: Starting task 4.0 in stage 18.0 (TID 87)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:37 INFO TaskSetManager: Finished task 1.0 in stage 17.0 (TID 79) in
3774 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:37 INFO TaskSetManager: Starting task 0.0 in stage 19.0 (TID 88)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:37 INFO TaskSetManager: Finished task 2.0 in stage 17.0 (TID 80) in
3725 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:37 INFO TaskSetManager: Starting task 1.0 in stage 19.0 (TID 89)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:37 INFO TaskSetManager: Finished task 3.0 in stage 17.0 (TID 81) in
3797 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:37 INFO TaskSetManager: Starting task 2.0 in stage 19.0 (TID 90)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:37 INFO TaskSetManager: Finished task 4.0 in stage 17.0 (TID 82) in
3801 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:37 INFO TaskSchedulerImpl: Removed TaskSet 17.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:37 INFO DAGScheduler: ShuffleMapStage 17 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 44.929 s
25/02/06 06:08:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:37 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 18, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:37 INFO DAGScheduler: failed: Set()
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:38 INFO TaskSetManager: Starting task 3.0 in stage 19.0 (TID 91)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:38 INFO TaskSetManager: Finished task 0.0 in stage 18.0 (TID 83) in
3592 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:38 INFO TaskSetManager: Starting task 4.0 in stage 19.0 (TID 92)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:38 INFO TaskSetManager: Finished task 1.0 in stage 18.0 (TID 84) in
3522 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:38 INFO TaskSetManager: Starting task 0.0 in stage 22.0 (TID 93)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:38 INFO TaskSetManager: Finished task 2.0 in stage 18.0 (TID 85) in
3537 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:39 INFO TaskSetManager: Starting task 1.0 in stage 22.0 (TID 94)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:39 INFO TaskSetManager: Finished task 3.0 in stage 18.0 (TID 86) in
3652 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:41 INFO TaskSetManager: Starting task 2.0 in stage 22.0 (TID 95)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:41 INFO TaskSetManager: Finished task 1.0 in stage 19.0 (TID 89) in
3545 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:41 INFO TaskSetManager: Starting task 3.0 in stage 22.0 (TID 96)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:41 INFO TaskSetManager: Finished task 4.0 in stage 18.0 (TID 87) in
3777 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:41 INFO TaskSchedulerImpl: Removed TaskSet 18.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:41 INFO DAGScheduler: ShuffleMapStage 18 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 48.425 s
25/02/06 06:08:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:41 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 19, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:41 INFO DAGScheduler: failed: Set()
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:41 INFO TaskSetManager: Starting task 4.0 in stage 22.0 (TID 97)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:41 INFO TaskSetManager: Finished task 0.0 in stage 19.0 (TID 88) in
3913 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:41 INFO TaskSetManager: Starting task 0.0 in stage 23.0 (TID 98)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:41 INFO TaskSetManager: Finished task 2.0 in stage 19.0 (TID 90) in
3746 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:41 INFO TaskSetManager: Starting task 1.0 in stage 23.0 (TID 99)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:41 INFO TaskSetManager: Finished task 0.0 in stage 22.0 (TID 93) in
3288 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:42 INFO TaskSetManager: Starting task 2.0 in stage 23.0 (TID 100)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:42 INFO TaskSetManager: Finished task 4.0 in stage 19.0 (TID 92) in
3549 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:42 INFO TaskSetManager: Starting task 3.0 in stage 23.0 (TID 101)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:42 INFO TaskSetManager: Finished task 3.0 in stage 19.0 (TID 91) in
3615 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:42 INFO TaskSchedulerImpl: Removed TaskSet 19.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:42 INFO DAGScheduler: ShuffleMapStage 19 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 49.275 s
25/02/06 06:08:42 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:42 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50,
ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 22, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29)
25/02/06 06:08:42 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:42 INFO DAGScheduler: failed: Set()
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:42 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:42 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:42 INFO TaskSetManager: Starting task 4.0 in stage 23.0 (TID 102)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:42 INFO TaskSetManager: Finished task 1.0 in stage 22.0 (TID 94) in
3312 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:44 INFO TaskSetManager: Starting task 0.0 in stage 24.0 (TID 103)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:44 INFO TaskSetManager: Finished task 2.0 in stage 22.0 (TID 95) in
3332 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:44 INFO TaskSetManager: Starting task 1.0 in stage 24.0 (TID 104)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:44 INFO TaskSetManager: Finished task 3.0 in stage 22.0 (TID 96) in
3384 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:44 INFO TaskSetManager: Starting task 2.0 in stage 24.0 (TID 105)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:44 INFO TaskSetManager: Finished task 4.0 in stage 22.0 (TID 97) in
3276 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:44 INFO TaskSchedulerImpl: Removed TaskSet 22.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:44 INFO DAGScheduler: ShuffleMapStage 22 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 51.815 s
25/02/06 06:08:44 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:44 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50,
ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 44, ShuffleMapStage 23, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:44 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:44 INFO DAGScheduler: failed: Set()
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:45 INFO TaskSetManager: Finished task 0.0 in stage 23.0 (TID 98) in
3700 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:45 INFO TaskSetManager: Starting task 3.0 in stage 24.0 (TID 106)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:45 INFO TaskSetManager: Starting task 4.0 in stage 24.0 (TID 107)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:45 INFO TaskSetManager: Finished task 2.0 in stage 23.0 (TID 100) in
3420 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:45 INFO TaskSetManager: Finished task 1.0 in stage 23.0 (TID 99) in
3577 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:45 INFO TaskSetManager: Starting task 0.0 in stage 25.0 (TID 108)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:45 INFO TaskSetManager: Starting task 1.0 in stage 25.0 (TID 109)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:45 INFO TaskSetManager: Finished task 3.0 in stage 23.0 (TID 101) in
3525 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:46 INFO TaskSetManager: Starting task 2.0 in stage 25.0 (TID 110)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:46 INFO TaskSetManager: Finished task 4.0 in stage 23.0 (TID 102) in
3653 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:46 INFO TaskSchedulerImpl: Removed TaskSet 23.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:46 INFO DAGScheduler: ShuffleMapStage 23 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 53.506 s
25/02/06 06:08:46 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:46 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50,
ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 24, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:46 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:46 INFO DAGScheduler: failed: Set()
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:47 INFO TaskSetManager: Starting task 3.0 in stage 25.0 (TID 111)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:47 INFO TaskSetManager: Finished task 0.0 in stage 24.0 (TID 103) in
3308 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:47 INFO TaskSetManager: Starting task 4.0 in stage 25.0 (TID 112)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:47 INFO TaskSetManager: Finished task 1.0 in stage 24.0 (TID 104) in
3188 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:48 INFO TaskSetManager: Starting task 0.0 in stage 26.0 (TID 113)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:48 INFO TaskSetManager: Finished task 2.0 in stage 24.0 (TID 105) in
3475 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:48 INFO TaskSetManager: Starting task 1.0 in stage 26.0 (TID 114)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:48 INFO TaskSetManager: Finished task 3.0 in stage 24.0 (TID 106) in
3515 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:48 INFO TaskSetManager: Starting task 2.0 in stage 26.0 (TID 115)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:48 INFO TaskSetManager: Finished task 4.0 in stage 24.0 (TID 107) in
3420 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:48 INFO TaskSchedulerImpl: Removed TaskSet 24.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:48 INFO DAGScheduler: ShuffleMapStage 24 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 56.005 s
25/02/06 06:08:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:48 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 25, ShuffleMapStage 77,
ShuffleMapStage 48, ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50,
ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 28, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 26, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:48 INFO DAGScheduler: failed: Set()
25/02/06 06:08:48 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:48 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:48 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:48 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:48 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:48 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:49 INFO TaskSetManager: Starting task 3.0 in stage 26.0 (TID 116)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:49 INFO TaskSetManager: Finished task 1.0 in stage 25.0 (TID 109) in
3396 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:49 INFO TaskSetManager: Starting task 4.0 in stage 26.0 (TID 117)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:49 INFO TaskSetManager: Finished task 0.0 in stage 25.0 (TID 108) in
3649 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:49 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:49 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:49 INFO TaskSetManager: Starting task 0.0 in stage 27.0 (TID 118)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:50 INFO TaskSetManager: Finished task 2.0 in stage 25.0 (TID 110) in
3599 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:51 INFO TaskSetManager: Starting task 1.0 in stage 27.0 (TID 119)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:51 INFO TaskSetManager: Finished task 4.0 in stage 25.0 (TID 112) in
3522 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:51 INFO TaskSetManager: Starting task 2.0 in stage 27.0 (TID 120)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:51 INFO TaskSetManager: Finished task 3.0 in stage 25.0 (TID 111) in
3650 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:51 INFO TaskSchedulerImpl: Removed TaskSet 25.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:51 INFO DAGScheduler: ShuffleMapStage 25 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 58.533 s
25/02/06 06:08:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:51 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 26,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58,
ShuffleMapStage 29)
25/02/06 06:08:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:51 INFO DAGScheduler: failed: Set()
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:51 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:51 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:51 INFO TaskSetManager: Starting task 3.0 in stage 27.0 (TID 121)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:51 INFO TaskSetManager: Finished task 0.0 in stage 26.0 (TID 113) in
3697 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:52 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:52 INFO TaskSetManager: Starting task 4.0 in stage 27.0 (TID 122)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:52 INFO TaskSetManager: Finished task 1.0 in stage 26.0 (TID 114) in
3693 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:52 INFO TaskSetManager: Starting task 0.0 in stage 29.0 (TID 123)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:52 INFO TaskSetManager: Finished task 2.0 in stage 26.0 (TID 115) in
3656 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:52 INFO TaskSetManager: Starting task 1.0 in stage 29.0 (TID 124)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:52 INFO TaskSetManager: Finished task 3.0 in stage 26.0 (TID 116) in
3553 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:52 INFO TaskSetManager: Starting task 2.0 in stage 29.0 (TID 125)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:52 INFO TaskSetManager: Finished task 4.0 in stage 26.0 (TID 117) in
3446 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:52 INFO TaskSchedulerImpl: Removed TaskSet 26.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:52 INFO DAGScheduler: ShuffleMapStage 26 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 59.670 s
25/02/06 06:08:52 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:52 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 27, ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:52 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:52 INFO DAGScheduler: failed: Set()
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:52 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:52 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:53 INFO TaskSetManager: Starting task 3.0 in stage 29.0 (TID 126)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:53 INFO TaskSetManager: Finished task 0.0 in stage 27.0 (TID 118) in
3439 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:55 INFO TaskSetManager: Starting task 4.0 in stage 29.0 (TID 127)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 1.0 in stage 27.0 (TID 119) in
3751 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:55 INFO TaskSetManager: Starting task 0.0 in stage 28.0 (TID 128)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 2.0 in stage 27.0 (TID 120) in
3662 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:08:55 INFO TaskSetManager: Starting task 1.0 in stage 28.0 (TID 129)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 3.0 in stage 27.0 (TID 121) in
3685 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:08:55 INFO TaskSetManager: Starting task 2.0 in stage 28.0 (TID 130)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 0.0 in stage 29.0 (TID 123) in
3227 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:08:55 INFO TaskSetManager: Starting task 3.0 in stage 28.0 (TID 131)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 1.0 in stage 29.0 (TID 124) in
3310 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:08:55 INFO TaskSetManager: Starting task 4.0 in stage 28.0 (TID 132)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:08:55 INFO TaskSetManager: Finished task 2.0 in stage 29.0 (TID 125) in
3318 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:56 INFO TaskSetManager: Starting task 0.0 in stage 30.0 (TID 133)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:08:56 INFO TaskSetManager: Finished task 4.0 in stage 27.0 (TID 122) in
3774 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:56 INFO TaskSchedulerImpl: Removed TaskSet 27.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:56 INFO DAGScheduler: ShuffleMapStage 27 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 63.245 s
25/02/06 06:08:56 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:56 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 29)
25/02/06 06:08:56 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:56 INFO DAGScheduler: failed: Set()
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:56 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:56 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:56 INFO TaskSetManager: Starting task 1.0 in stage 30.0 (TID 134)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:56 INFO TaskSetManager: Finished task 3.0 in stage 29.0 (TID 126) in
3325 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:08:58 INFO TaskSetManager: Starting task 2.0 in stage 30.0 (TID 135)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:08:58 INFO TaskSetManager: Finished task 4.0 in stage 29.0 (TID 127) in
3355 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:08:58 INFO TaskSchedulerImpl: Removed TaskSet 29.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:58 INFO DAGScheduler: ShuffleMapStage 29 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 65.512 s
25/02/06 06:08:58 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:58 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 28, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:08:58 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:58 INFO DAGScheduler: failed: Set()
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:58 INFO TaskSetManager: Starting task 3.0 in stage 30.0 (TID 136)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:58 INFO TaskSetManager: Finished task 0.0 in stage 28.0 (TID 128) in
3414 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:58 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:58 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:59 INFO TaskSetManager: Starting task 4.0 in stage 30.0 (TID 137)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:08:59 INFO TaskSetManager: Finished task 1.0 in stage 28.0 (TID 129) in
3503 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:08:59 INFO TaskSetManager: Finished task 3.0 in stage 28.0 (TID 131) in
3347 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:08:59 INFO TaskSetManager: Starting task 0.0 in stage 31.0 (TID 138)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:08:59 INFO TaskSetManager: Starting task 1.0 in stage 31.0 (TID 139)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:08:59 INFO TaskSetManager: Finished task 4.0 in stage 28.0 (TID 132) in
3432 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:08:59 INFO TaskSetManager: Starting task 2.0 in stage 31.0 (TID 140)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:08:59 INFO TaskSetManager: Finished task 2.0 in stage 28.0 (TID 130) in
3703 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:08:59 INFO TaskSchedulerImpl: Removed TaskSet 28.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:08:59 INFO DAGScheduler: ShuffleMapStage 28 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 66.567 s
25/02/06 06:08:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:08:59 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 30, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 32, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58)
25/02/06 06:08:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:08:59 INFO DAGScheduler: failed: Set()
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:08:59 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:08:59 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:08:59 INFO TaskSetManager: Starting task 3.0 in stage 31.0 (TID 141)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:08:59 INFO TaskSetManager: Finished task 0.0 in stage 30.0 (TID 133) in
3669 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:00 INFO TaskSetManager: Starting task 4.0 in stage 31.0 (TID 142)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:00 INFO TaskSetManager: Finished task 1.0 in stage 30.0 (TID 134) in
3775 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:01 INFO TaskSetManager: Starting task 0.0 in stage 32.0 (TID 143)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:01 INFO TaskSetManager: Finished task 2.0 in stage 30.0 (TID 135) in
3440 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:02 INFO TaskSetManager: Starting task 1.0 in stage 32.0 (TID 144)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:02 INFO TaskSetManager: Finished task 3.0 in stage 30.0 (TID 136) in
3620 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:02 INFO TaskSetManager: Starting task 2.0 in stage 32.0 (TID 145)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:02 INFO TaskSetManager: Finished task 0.0 in stage 31.0 (TID 138) in
3210 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:02 INFO TaskSetManager: Starting task 3.0 in stage 32.0 (TID 146)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:02 INFO TaskSetManager: Finished task 1.0 in stage 31.0 (TID 139) in
3288 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:02 INFO TaskSetManager: Starting task 4.0 in stage 32.0 (TID 147)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:02 INFO TaskSetManager: Finished task 2.0 in stage 31.0 (TID 140) in
3413 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:02 INFO TaskSetManager: Starting task 0.0 in stage 33.0 (TID 148)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:02 INFO TaskSetManager: Finished task 4.0 in stage 30.0 (TID 137) in
3918 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:02 INFO TaskSchedulerImpl: Removed TaskSet 30.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:02 INFO DAGScheduler: ShuffleMapStage 30 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 70.004 s
25/02/06 06:09:02 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:02 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 31, ShuffleMapStage 83,
ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 36, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 32, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 33, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:02 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:02 INFO DAGScheduler: failed: Set()
25/02/06 06:09:03 INFO TaskSetManager: Starting task 1.0 in stage 33.0 (TID 149)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:03 INFO TaskSetManager: Finished task 3.0 in stage 31.0 (TID 141) in
3173 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:03 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:03 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:03 INFO TaskSetManager: Starting task 2.0 in stage 33.0 (TID 150)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:03 INFO TaskSetManager: Finished task 4.0 in stage 31.0 (TID 142) in
3261 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:03 INFO TaskSchedulerImpl: Removed TaskSet 31.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:03 INFO DAGScheduler: ShuffleMapStage 31 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 70.831 s
25/02/06 06:09:03 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:03 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 32,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:03 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:03 INFO DAGScheduler: failed: Set()
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:04 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:04 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:05 INFO TaskSetManager: Starting task 3.0 in stage 33.0 (TID 151)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:05 INFO TaskSetManager: Finished task 0.0 in stage 32.0 (TID 143) in
3666 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:05 INFO TaskSetManager: Starting task 4.0 in stage 33.0 (TID 152)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:05 INFO TaskSetManager: Finished task 1.0 in stage 32.0 (TID 144) in
3546 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:05 INFO TaskSetManager: Starting task 0.0 in stage 34.0 (TID 153)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:05 INFO TaskSetManager: Finished task 2.0 in stage 32.0 (TID 145) in
3505 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:06 INFO TaskSetManager: Starting task 1.0 in stage 34.0 (TID 154)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:06 INFO TaskSetManager: Finished task 3.0 in stage 32.0 (TID 146) in
3639 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:06 INFO TaskSetManager: Starting task 2.0 in stage 34.0 (TID 155)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:06 INFO TaskSetManager: Finished task 4.0 in stage 32.0 (TID 147) in
3708 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:06 INFO TaskSchedulerImpl: Removed TaskSet 32.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:06 INFO DAGScheduler: ShuffleMapStage 32 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 73.641 s
25/02/06 06:09:06 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:06 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 33, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:06 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:06 INFO DAGScheduler: failed: Set()
25/02/06 06:09:06 INFO TaskSetManager: Starting task 3.0 in stage 34.0 (TID 156)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:06 INFO TaskSetManager: Finished task 0.0 in stage 33.0 (TID 148) in
3655 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:06 INFO TaskSetManager: Starting task 4.0 in stage 34.0 (TID 157)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:06 INFO TaskSetManager: Finished task 1.0 in stage 33.0 (TID 149) in
3679 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:06 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:06 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:07 INFO TaskSetManager: Starting task 0.0 in stage 35.0 (TID 158)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:07 INFO TaskSetManager: Finished task 2.0 in stage 33.0 (TID 150) in
3471 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:09 INFO TaskSetManager: Starting task 1.0 in stage 35.0 (TID 159)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:09 INFO TaskSetManager: Finished task 3.0 in stage 33.0 (TID 151) in
3641 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:09 INFO TaskSetManager: Starting task 2.0 in stage 35.0 (TID 160)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:09 INFO TaskSetManager: Finished task 4.0 in stage 33.0 (TID 152) in
3637 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:09 INFO TaskSchedulerImpl: Removed TaskSet 33.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:09 INFO DAGScheduler: ShuffleMapStage 33 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 76.320 s
25/02/06 06:09:09 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:09 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 34, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 36, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58)
25/02/06 06:09:09 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:09 INFO DAGScheduler: failed: Set()
25/02/06 06:09:09 INFO TaskSetManager: Starting task 3.0 in stage 35.0 (TID 161)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:09 INFO TaskSetManager: Finished task 0.0 in stage 34.0 (TID 153) in
3426 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:09 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:09 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:09 INFO TaskSetManager: Starting task 4.0 in stage 35.0 (TID 162)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:09 INFO TaskSetManager: Finished task 1.0 in stage 34.0 (TID 154) in
3572 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:10 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:10 INFO TaskSetManager: Starting task 0.0 in stage 36.0 (TID 163)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:10 INFO TaskSetManager: Finished task 3.0 in stage 34.0 (TID 156) in
3626 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:10 INFO TaskSetManager: Starting task 1.0 in stage 36.0 (TID 164)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:10 INFO TaskSetManager: Finished task 2.0 in stage 34.0 (TID 155) in
3683 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:10 INFO TaskSetManager: Starting task 2.0 in stage 36.0 (TID 165)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:10 INFO TaskSetManager: Finished task 4.0 in stage 34.0 (TID 157) in
3676 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:10 INFO TaskSchedulerImpl: Removed TaskSet 34.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:10 INFO DAGScheduler: ShuffleMapStage 34 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 77.350 s
25/02/06 06:09:10 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:10 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 35, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:10 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:10 INFO DAGScheduler: failed: Set()
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:10 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:10 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:10 INFO TaskSetManager: Starting task 3.0 in stage 36.0 (TID 166)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:10 INFO TaskSetManager: Finished task 0.0 in stage 35.0 (TID 158) in
3466 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:12 INFO TaskSetManager: Starting task 4.0 in stage 36.0 (TID 167)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:12 INFO TaskSetManager: Finished task 1.0 in stage 35.0 (TID 159) in
3476 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:12 INFO TaskSetManager: Starting task 0.0 in stage 37.0 (TID 168)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:12 INFO TaskSetManager: Finished task 3.0 in stage 35.0 (TID 161) in
3444 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:12 INFO TaskSetManager: Starting task 1.0 in stage 37.0 (TID 169)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:12 INFO TaskSetManager: Finished task 2.0 in stage 35.0 (TID 160) in
3536 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:13 INFO TaskSetManager: Starting task 2.0 in stage 37.0 (TID 170)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:13 INFO TaskSetManager: Finished task 4.0 in stage 35.0 (TID 162) in
3448 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:13 INFO TaskSchedulerImpl: Removed TaskSet 35.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:13 INFO DAGScheduler: ShuffleMapStage 35 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 80.259 s
25/02/06 06:09:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:13 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 36,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:13 INFO DAGScheduler: failed: Set()
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:13 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:13 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:13 INFO TaskSetManager: Starting task 3.0 in stage 37.0 (TID 171)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:13 INFO TaskSetManager: Finished task 0.0 in stage 36.0 (TID 163) in
3569 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:13 INFO TaskSetManager: Starting task 4.0 in stage 37.0 (TID 172)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:13 INFO TaskSetManager: Finished task 1.0 in stage 36.0 (TID 164) in
3693 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:13 INFO TaskSetManager: Starting task 0.0 in stage 38.0 (TID 173)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:13 INFO TaskSetManager: Finished task 2.0 in stage 36.0 (TID 165) in
3629 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:14 INFO TaskSetManager: Starting task 1.0 in stage 38.0 (TID 174)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:14 INFO TaskSetManager: Finished task 3.0 in stage 36.0 (TID 166) in
3655 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:16 INFO TaskSetManager: Starting task 2.0 in stage 38.0 (TID 175)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:16 INFO TaskSetManager: Finished task 1.0 in stage 37.0 (TID 169) in
3252 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:16 INFO TaskSetManager: Starting task 0.0 in stage 39.0 (TID 176)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:16 INFO TaskSetManager: Finished task 0.0 in stage 37.0 (TID 168) in
3346 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:16 INFO TaskSetManager: Starting task 1.0 in stage 39.0 (TID 177)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:16 INFO TaskSetManager: Finished task 4.0 in stage 36.0 (TID 167) in
3691 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:16 INFO TaskSchedulerImpl: Removed TaskSet 36.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:16 INFO DAGScheduler: ShuffleMapStage 36 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 83.310 s
25/02/06 06:09:16 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:16 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 37, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 39, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:16 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:16 INFO DAGScheduler: failed: Set()
25/02/06 06:09:16 INFO TaskSetManager: Starting task 2.0 in stage 39.0 (TID 178)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:16 INFO TaskSetManager: Finished task 2.0 in stage 37.0 (TID 170) in
3153 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:16 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:16 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:16 INFO TaskSetManager: Starting task 3.0 in stage 39.0 (TID 179)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:16 INFO TaskSetManager: Finished task 3.0 in stage 37.0 (TID 171) in
2864 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:17 INFO TaskSetManager: Starting task 4.0 in stage 39.0 (TID 180)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:17 INFO TaskSetManager: Finished task 4.0 in stage 37.0 (TID 172) in
3349 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:17 INFO TaskSchedulerImpl: Removed TaskSet 37.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:17 INFO DAGScheduler: ShuffleMapStage 37 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 84.280 s
25/02/06 06:09:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:17 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 39,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 87, ShuffleMapStage 79,
ShuffleMapStage 58)
25/02/06 06:09:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:17 INFO DAGScheduler: failed: Set()
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:17 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:17 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:19 INFO TaskSetManager: Starting task 0.0 in stage 40.0 (TID 181)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Finished task 1.0 in stage 39.0 (TID 177) in
2881 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:19 INFO TaskSetManager: Finished task 0.0 in stage 39.0 (TID 176) in
3140 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:19 INFO TaskSetManager: Starting task 1.0 in stage 40.0 (TID 182)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Starting task 2.0 in stage 40.0 (TID 183)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Finished task 3.0 in stage 39.0 (TID 179) in
2925 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:19 INFO TaskSetManager: Starting task 3.0 in stage 40.0 (TID 184)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Finished task 0.0 in stage 38.0 (TID 173) in
5725 ms on 10.183.229.123 (executor 0) (1/3)
25/02/06 06:09:19 INFO TaskSetManager: Starting task 4.0 in stage 40.0 (TID 185)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Finished task 2.0 in stage 39.0 (TID 178) in
3393 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:19 INFO TaskSetManager: Starting task 0.0 in stage 41.0 (TID 186)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:19 INFO TaskSetManager: Finished task 1.0 in stage 38.0 (TID 174) in
5563 ms on 10.183.232.45 (executor 2) (2/3)
25/02/06 06:09:20 INFO TaskSetManager: Starting task 1.0 in stage 41.0 (TID 187)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:20 INFO TaskSetManager: Finished task 4.0 in stage 39.0 (TID 180) in
3273 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:20 INFO TaskSchedulerImpl: Removed TaskSet 39.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:20 INFO DAGScheduler: ShuffleMapStage 39 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 87.516 s
25/02/06 06:09:20 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:20 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 38, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 41, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 56, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:20 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:20 INFO DAGScheduler: failed: Set()
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:20 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:20 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:21 INFO TaskSetManager: Finished task 2.0 in stage 38.0 (TID 175) in
5378 ms on 10.183.229.123 (executor 0) (3/3)
25/02/06 06:09:21 INFO TaskSchedulerImpl: Removed TaskSet 38.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:21 INFO DAGScheduler: ShuffleMapStage 38 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 88.435 s
25/02/06 06:09:21 INFO TaskSetManager: Starting task 2.0 in stage 41.0 (TID 188)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:21 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:21 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 40, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 41, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:21 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:21 INFO DAGScheduler: failed: Set()
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:21 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:21 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(1, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(15, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(6, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(45, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(36, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(24, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(28, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(33, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(16, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(20, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(35, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(19, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(17, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(14, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(10, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(32, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(21, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(18, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(5, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(13, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(43, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(12, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(3, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(9, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(2, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(4, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(41, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(37, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(47, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(42, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(38, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(25, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(26, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(46, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO ShufflePartitionsUtil: For shuffle(44, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:22 INFO TaskSetManager: Starting task 3.0 in stage 41.0 (TID 189)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:22 INFO TaskSetManager: Finished task 2.0 in stage 40.0 (TID 183) in
3191 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:22 INFO TaskSetManager: Finished task 0.0 in stage 40.0 (TID 181) in
3591 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:22 INFO TaskSetManager: Starting task 4.0 in stage 41.0 (TID 190)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:23 INFO TaskSetManager: Starting task 0.0 in stage 42.0 (TID 191)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:23 INFO TaskSetManager: Finished task 1.0 in stage 40.0 (TID 182) in
3917 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:23 INFO TaskSetManager: Starting task 1.0 in stage 42.0 (TID 192)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:23 INFO TaskSetManager: Finished task 4.0 in stage 40.0 (TID 185) in
3492 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:23 INFO TaskSetManager: Finished task 3.0 in stage 40.0 (TID 184) in
3853 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:23 INFO DAGScheduler: ShuffleMapStage 40 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 90.461 s
25/02/06 06:09:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:23 INFO DAGScheduler: running: Set(ShuffleMapStage 81,
ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 83, ShuffleMapStage 75,
ShuffleMapStage 54, ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 71,
ShuffleMapStage 50, ShuffleMapStage 42, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 44, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 43,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 41,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 56, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58)
25/02/06 06:09:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:23 INFO DAGScheduler: failed: Set()
25/02/06 06:09:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:09:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:09:23 INFO TaskSchedulerImpl: Removed TaskSet 40.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:23 INFO TaskSetManager: Starting task 2.0 in stage 42.0 (TID 193)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:23 INFO TaskSetManager: Finished task 0.0 in stage 41.0 (TID 186) in
3649 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:23 INFO TaskSetManager: Starting task 3.0 in stage 42.0 (TID 194)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:23 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098318) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098318)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 564 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 102
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 95 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 98
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
96, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 98
(MapPartitionsRDD[564] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 582 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 103
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 96 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 100
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
99, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 100
(MapPartitionsRDD[582] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 577 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 101
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 97 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 102
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
101, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 102
(MapPartitionsRDD[577] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 595 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 106
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 98 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 104
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
103, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 104
(MapPartitionsRDD[595] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 599 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 108
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 99 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 106
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
105, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 106
(MapPartitionsRDD[599] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 596 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 107
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 100 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 108
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
107, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 108
(MapPartitionsRDD[596] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 563 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 98
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 101 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 110
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 109)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 110
(MapPartitionsRDD[563] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 567 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 99
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 102 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 112
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
111, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 112
(MapPartitionsRDD[567] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 537 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 93
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 103 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 114
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 113)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 114
(MapPartitionsRDD[537] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 593 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 105
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 104 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 116
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
115, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 116
(MapPartitionsRDD[593] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 554 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 96
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 106 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 118
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
117, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 118
(MapPartitionsRDD[554] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 546 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 94
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 107 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 120
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
119, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 120
(MapPartitionsRDD[546] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 591 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 104
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 108 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 122
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
121, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 122
(MapPartitionsRDD[591] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 547 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 95
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 105 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 124
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
123, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 124
(MapPartitionsRDD[547] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 570 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 100
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 109 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 126
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
125, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 126
(MapPartitionsRDD[570] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 623 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 111
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 111 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 128
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 127)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 128
(MapPartitionsRDD[623] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 622 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 110
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 110 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 130
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
129, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 130
(MapPartitionsRDD[622] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 555 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 97
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 112 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 132
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 131)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 132
(MapPartitionsRDD[555] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 104 (MapPartitionsRDD[595] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 104.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 104.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_104.0 tasks
to pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 102 (MapPartitionsRDD[577] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 102.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 102.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_102.0 tasks
to pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 100 (MapPartitionsRDD[582] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 100.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 100.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_100.0 tasks
to pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 106 (MapPartitionsRDD[599] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 106.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 106.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_106.0 tasks
to pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 98 (MapPartitionsRDD[564] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 98.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 98.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_98.0 tasks to
pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 108 (MapPartitionsRDD[596] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 108.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 108.0 using PreferredLocationsV1
25/02/06 06:09:23 INFO FairSchedulableBuilder: Added task set TaskSet_108.0 tasks
to pool 5400065285585087590
25/02/06 06:09:23 INFO DAGScheduler: Registering RDD 610 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 109
25/02/06 06:09:23 INFO DAGScheduler: Got map stage job 113 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:23 INFO DAGScheduler: Final stage: ShuffleMapStage 134
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
133, ShuffleMapStage 97)
25/02/06 06:09:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:23 INFO DAGScheduler: Submitting ShuffleMapStage 134
(MapPartitionsRDD[610] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 110 (MapPartitionsRDD[563] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:23 INFO TaskSchedulerImpl: Adding task set 110.0 with 9 tasks
resource profile 0
25/02/06 06:09:23 INFO TaskSetManager: TaskSet 110.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_110.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 112 (MapPartitionsRDD[567] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 112.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 112.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_112.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 114 (MapPartitionsRDD[537] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 114.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 114.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_114.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 637 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 112
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 114 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 136
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
135, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 136
(MapPartitionsRDD[637] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 116 (MapPartitionsRDD[593] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 116.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 116.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_116.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 638 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 113
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 115 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 138
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
137, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 138
(MapPartitionsRDD[638] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 118 (MapPartitionsRDD[554] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 118.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 118.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_118.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 120 (MapPartitionsRDD[546] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 120.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 120.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_120.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 650 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 114
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 116 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 140
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
139, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 140
(MapPartitionsRDD[650] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 122 (MapPartitionsRDD[591] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 122.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 122.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_122.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 662 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 115
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 118 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 142
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
141, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 142
(MapPartitionsRDD[662] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 124 (MapPartitionsRDD[547] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 124.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 124.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_124.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 126 (MapPartitionsRDD[570] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 126.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 126.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_126.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 659 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 116
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 117 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 144
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
143, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 144
(MapPartitionsRDD[659] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 670 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 117
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 119 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 146
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 145)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 146
(MapPartitionsRDD[670] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 128 (MapPartitionsRDD[623] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 128.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 128.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_128.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 130 (MapPartitionsRDD[622] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 130.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 130.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_130.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 681 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 118
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 120 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 148
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
147, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 148
(MapPartitionsRDD[681] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 132 (MapPartitionsRDD[555] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 132.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 132.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_132.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 686 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 119
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 121 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 150
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 149)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 150
(MapPartitionsRDD[686] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 694 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 120
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 122 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 152
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
151, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 152
(MapPartitionsRDD[694] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 702 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 121
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 123 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 154
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
153, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 154
(MapPartitionsRDD[702] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 710 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 122
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 124 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 156
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
155, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 156
(MapPartitionsRDD[710] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 134 (MapPartitionsRDD[610] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 134.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 134.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_134.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 136 (MapPartitionsRDD[637] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 136.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 136.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_136.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 718 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 123
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 125 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 158
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
157, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 158
(MapPartitionsRDD[718] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 138 (MapPartitionsRDD[638] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 138.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 138.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_138.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 726 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 124
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 126 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 160
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
159, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 160
(MapPartitionsRDD[726] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 734 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 125
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 127 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 162
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
161, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 162
(MapPartitionsRDD[734] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 140 (MapPartitionsRDD[650] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 140.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 140.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_140.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 142 (MapPartitionsRDD[662] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 142.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 142.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_142.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 742 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 126
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 128 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 164
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 163)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 164
(MapPartitionsRDD[742] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Registering RDD 750 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 127
25/02/06 06:09:24 INFO DAGScheduler: Got map stage job 129 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:24 INFO DAGScheduler: Final stage: ShuffleMapStage 166
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
165, ShuffleMapStage 97)
25/02/06 06:09:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:24 INFO DAGScheduler: Submitting ShuffleMapStage 166
(MapPartitionsRDD[750] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 144 (MapPartitionsRDD[659] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 144.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 144.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_144.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 146 (MapPartitionsRDD[670] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 146.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 146.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_146.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 148 (MapPartitionsRDD[681] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 148.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 148.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_148.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 150 (MapPartitionsRDD[686] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 150.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 150.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_150.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 152 (MapPartitionsRDD[694] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 152.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 152.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_152.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 154 (MapPartitionsRDD[702] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 154.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 154.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_154.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 156 (MapPartitionsRDD[710] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 156.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 156.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_156.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 158 (MapPartitionsRDD[718] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 158.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 158.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_158.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 160 (MapPartitionsRDD[726] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 160.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 160.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_160.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 162 (MapPartitionsRDD[734] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 162.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 162.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_162.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 164 (MapPartitionsRDD[742] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 164.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 164.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_164.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 166 (MapPartitionsRDD[750] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:24 INFO TaskSchedulerImpl: Adding task set 166.0 with 9 tasks
resource profile 0
25/02/06 06:09:24 INFO TaskSetManager: TaskSet 166.0 using PreferredLocationsV1
25/02/06 06:09:24 INFO FairSchedulableBuilder: Added task set TaskSet_166.0 tasks
to pool 5400065285585087590
25/02/06 06:09:24 INFO TaskSetManager: Finished task 1.0 in stage 41.0 (TID 187) in
3784 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:24 INFO TaskSetManager: Starting task 4.0 in stage 42.0 (TID 195)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:24 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:24 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:25 INFO ShufflePartitionsUtil: For shuffle(49, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:25 INFO TaskSetManager: Starting task 0.0 in stage 43.0 (TID 196)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:25 INFO TaskSetManager: Finished task 2.0 in stage 41.0 (TID 188) in
3869 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:25 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099312) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099312)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:25 INFO DAGScheduler: Registering RDD 758 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 128
25/02/06 06:09:25 INFO DAGScheduler: Got map stage job 130 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:25 INFO DAGScheduler: Final stage: ShuffleMapStage 168
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 167)
25/02/06 06:09:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:25 INFO DAGScheduler: Submitting ShuffleMapStage 168
(MapPartitionsRDD[758] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:25 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 168 (MapPartitionsRDD[758] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:25 INFO TaskSchedulerImpl: Adding task set 168.0 with 9 tasks
resource profile 0
25/02/06 06:09:25 INFO TaskSetManager: TaskSet 168.0 using PreferredLocationsV1
25/02/06 06:09:25 INFO FairSchedulableBuilder: Added task set TaskSet_168.0 tasks
to pool 5400065285585087590
25/02/06 06:09:26 INFO TaskSetManager: Starting task 1.0 in stage 43.0 (TID 197)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:26 INFO TaskSetManager: Finished task 4.0 in stage 41.0 (TID 190) in
3640 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:26 INFO TaskSetManager: Starting task 2.0 in stage 43.0 (TID 198)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:26 INFO TaskSetManager: Finished task 3.0 in stage 41.0 (TID 189) in
3871 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:26 INFO TaskSchedulerImpl: Removed TaskSet 41.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:26 INFO DAGScheduler: ShuffleMapStage 41 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 93.549 s
25/02/06 06:09:26 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:26 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 42,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 44, ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 46, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 126, ShuffleMapStage 76,
ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 43, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:26 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:26 INFO DAGScheduler: failed: Set()
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:26 INFO TaskSetManager: Starting task 3.0 in stage 43.0 (TID 199)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:26 INFO TaskSetManager: Finished task 0.0 in stage 42.0 (TID 191) in
3659 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:26 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:26 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:26 INFO TaskSetManager: Starting task 4.0 in stage 43.0 (TID 200)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:26 INFO TaskSetManager: Finished task 1.0 in stage 42.0 (TID 192) in
3634 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:26 INFO TaskSetManager: Starting task 0.0 in stage 44.0 (TID 201)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:26 INFO TaskSetManager: Finished task 3.0 in stage 42.0 (TID 194) in
3364 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:27 INFO TaskSetManager: Starting task 1.0 in stage 44.0 (TID 202)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:27 INFO TaskSetManager: Finished task 2.0 in stage 42.0 (TID 193) in
3633 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:27 INFO ShufflePartitionsUtil: For shuffle(50, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:28 INFO DAGScheduler: Registering RDD 766 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 129
25/02/06 06:09:28 INFO DAGScheduler: Got map stage job 131 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:28 INFO DAGScheduler: Final stage: ShuffleMapStage 170
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:28 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
169, ShuffleMapStage 97)
25/02/06 06:09:28 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:28 INFO DAGScheduler: Submitting ShuffleMapStage 170
(MapPartitionsRDD[766] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:28 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099437) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099437)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:28 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 170 (MapPartitionsRDD[766] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:28 INFO TaskSchedulerImpl: Adding task set 170.0 with 9 tasks
resource profile 0
25/02/06 06:09:28 INFO TaskSetManager: TaskSet 170.0 using PreferredLocationsV1
25/02/06 06:09:28 INFO FairSchedulableBuilder: Added task set TaskSet_170.0 tasks
to pool 5400065285585087590
25/02/06 06:09:28 INFO TaskSetManager: Starting task 2.0 in stage 44.0 (TID 203)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:28 INFO TaskSetManager: Finished task 4.0 in stage 42.0 (TID 195) in
3876 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:28 INFO TaskSchedulerImpl: Removed TaskSet 42.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:28 INFO DAGScheduler: ShuffleMapStage 42 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 95.118 s
25/02/06 06:09:28 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:28 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 43, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:28 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:28 INFO DAGScheduler: failed: Set()
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:28 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:28 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:28 INFO TaskSetManager: Starting task 3.0 in stage 44.0 (TID 204)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:28 INFO TaskSetManager: Finished task 0.0 in stage 43.0 (TID 196) in
3395 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:29 INFO ShufflePartitionsUtil: For shuffle(7, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:29 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098543) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098543)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:29 INFO DAGScheduler: Registering RDD 774 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 130
25/02/06 06:09:29 INFO DAGScheduler: Got map stage job 132 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:29 INFO DAGScheduler: Final stage: ShuffleMapStage 172
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
171, ShuffleMapStage 97)
25/02/06 06:09:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:29 INFO DAGScheduler: Submitting ShuffleMapStage 172
(MapPartitionsRDD[774] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:29 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 172 (MapPartitionsRDD[774] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:29 INFO TaskSchedulerImpl: Adding task set 172.0 with 9 tasks
resource profile 0
25/02/06 06:09:29 INFO TaskSetManager: TaskSet 172.0 using PreferredLocationsV1
25/02/06 06:09:29 INFO FairSchedulableBuilder: Added task set TaskSet_172.0 tasks
to pool 5400065285585087590
25/02/06 06:09:30 INFO TaskSetManager: Starting task 4.0 in stage 44.0 (TID 205)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Finished task 2.0 in stage 43.0 (TID 198) in
3526 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:30 INFO TaskSetManager: Finished task 3.0 in stage 43.0 (TID 199) in
3323 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:30 INFO TaskSetManager: Starting task 0.0 in stage 46.0 (TID 206)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Starting task 1.0 in stage 46.0 (TID 207)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Finished task 1.0 in stage 43.0 (TID 197) in
3740 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:30 INFO TaskSetManager: Starting task 2.0 in stage 46.0 (TID 208)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Finished task 4.0 in stage 43.0 (TID 200) in
3277 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:30 INFO TaskSchedulerImpl: Removed TaskSet 43.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:30 INFO DAGScheduler: ShuffleMapStage 43 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 97.066 s
25/02/06 06:09:30 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:30 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 44,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 74,
ShuffleMapStage 53, ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 47, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:30 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:30 INFO DAGScheduler: failed: Set()
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:30 INFO TaskSetManager: Starting task 3.0 in stage 46.0 (TID 209)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Finished task 0.0 in stage 44.0 (TID 201) in
3472 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:30 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:30 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:30 INFO ShufflePartitionsUtil: For shuffle(30, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:30 INFO TaskSetManager: Starting task 4.0 in stage 46.0 (TID 210)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:30 INFO TaskSetManager: Finished task 1.0 in stage 44.0 (TID 202) in
3794 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:31 INFO TaskSetManager: Starting task 0.0 in stage 45.0 (TID 211)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:31 INFO TaskSetManager: Finished task 2.0 in stage 44.0 (TID 203) in
3481 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:31 INFO DAGScheduler: Registering RDD 782 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 131
25/02/06 06:09:31 INFO DAGScheduler: Got map stage job 133 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:31 INFO DAGScheduler: Final stage: ShuffleMapStage 174
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
173, ShuffleMapStage 97)
25/02/06 06:09:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:31 INFO DAGScheduler: Submitting ShuffleMapStage 174
(MapPartitionsRDD[782] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:31 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 174 (MapPartitionsRDD[782] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:31 INFO TaskSchedulerImpl: Adding task set 174.0 with 9 tasks
resource profile 0
25/02/06 06:09:31 INFO TaskSetManager: TaskSet 174.0 using PreferredLocationsV1
25/02/06 06:09:31 INFO FairSchedulableBuilder: Added task set TaskSet_174.0 tasks
to pool 5400065285585087590
25/02/06 06:09:31 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100646) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100646)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:32 INFO TaskSetManager: Starting task 1.0 in stage 45.0 (TID 212)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:32 INFO TaskSetManager: Finished task 3.0 in stage 44.0 (TID 204) in
3631 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:33 INFO TaskSetManager: Starting task 2.0 in stage 45.0 (TID 213)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:33 INFO TaskSetManager: Finished task 4.0 in stage 44.0 (TID 205) in
3498 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:33 INFO TaskSchedulerImpl: Removed TaskSet 44.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:33 INFO DAGScheduler: ShuffleMapStage 44 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 100.517 s
25/02/06 06:09:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:33 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 146,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 46, ShuffleMapStage 140,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 138,
ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51,
ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 45, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:33 INFO DAGScheduler: failed: Set()
25/02/06 06:09:33 INFO TaskSetManager: Starting task 3.0 in stage 45.0 (TID 214)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:33 INFO TaskSetManager: Finished task 2.0 in stage 46.0 (TID 208) in
3476 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:33 INFO TaskSetManager: Starting task 4.0 in stage 45.0 (TID 215)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:33 INFO TaskSetManager: Finished task 0.0 in stage 46.0 (TID 206) in
3636 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:33 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:33 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:33 INFO TaskSetManager: Starting task 0.0 in stage 47.0 (TID 216)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:33 INFO TaskSetManager: Finished task 1.0 in stage 46.0 (TID 207) in
3747 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:34 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:34 INFO TaskSetManager: Starting task 1.0 in stage 47.0 (TID 217)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:34 INFO TaskSetManager: Finished task 3.0 in stage 46.0 (TID 209) in
3710 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:34 INFO ShufflePartitionsUtil: For shuffle(23, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:34 INFO TaskSetManager: Starting task 2.0 in stage 47.0 (TID 218)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:34 INFO TaskSetManager: Finished task 4.0 in stage 46.0 (TID 210) in
3646 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:34 INFO TaskSchedulerImpl: Removed TaskSet 46.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:34 INFO DAGScheduler: ShuffleMapStage 46 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 101.420 s
25/02/06 06:09:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:34 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 146,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 47, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 70, ShuffleMapStage 49, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 45, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166,
ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:09:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:34 INFO DAGScheduler: failed: Set()
25/02/06 06:09:34 INFO TaskSetManager: Starting task 3.0 in stage 47.0 (TID 219)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:34 INFO TaskSetManager: Finished task 0.0 in stage 45.0 (TID 211) in
3140 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:35 INFO DAGScheduler: Registering RDD 790 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 132
25/02/06 06:09:35 INFO DAGScheduler: Got map stage job 134 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:35 INFO DAGScheduler: Final stage: ShuffleMapStage 176
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:35 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 175)
25/02/06 06:09:35 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:35 INFO DAGScheduler: Submitting ShuffleMapStage 176
(MapPartitionsRDD[790] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:35 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 176 (MapPartitionsRDD[790] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:35 INFO TaskSchedulerImpl: Adding task set 176.0 with 9 tasks
resource profile 0
25/02/06 06:09:35 INFO TaskSetManager: TaskSet 176.0 using PreferredLocationsV1
25/02/06 06:09:35 INFO FairSchedulableBuilder: Added task set TaskSet_176.0 tasks
to pool 5400065285585087590
25/02/06 06:09:35 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100707) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100707)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:35 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:35 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:35 INFO TaskSetManager: Starting task 4.0 in stage 47.0 (TID 220)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:35 INFO TaskSetManager: Finished task 1.0 in stage 45.0 (TID 212) in
3063 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:35 INFO ShufflePartitionsUtil: For shuffle(40, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:36 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099499) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099499)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:36 INFO DAGScheduler: Registering RDD 798 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 133
25/02/06 06:09:36 INFO DAGScheduler: Got map stage job 135 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:36 INFO DAGScheduler: Final stage: ShuffleMapStage 178
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
177, ShuffleMapStage 97)
25/02/06 06:09:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:36 INFO DAGScheduler: Submitting ShuffleMapStage 178
(MapPartitionsRDD[798] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:36 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 178 (MapPartitionsRDD[798] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:36 INFO TaskSchedulerImpl: Adding task set 178.0 with 9 tasks
resource profile 0
25/02/06 06:09:36 INFO TaskSetManager: TaskSet 178.0 using PreferredLocationsV1
25/02/06 06:09:36 INFO FairSchedulableBuilder: Added task set TaskSet_178.0 tasks
to pool 5400065285585087590
25/02/06 06:09:36 INFO TaskSetManager: Starting task 0.0 in stage 50.0 (TID 221)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:36 INFO TaskSetManager: Finished task 3.0 in stage 45.0 (TID 214) in
3122 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:37 INFO TaskSetManager: Starting task 1.0 in stage 50.0 (TID 222)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:37 INFO TaskSetManager: Finished task 2.0 in stage 45.0 (TID 213) in
3468 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:37 INFO TaskSetManager: Starting task 2.0 in stage 50.0 (TID 223)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:37 INFO TaskSetManager: Finished task 0.0 in stage 47.0 (TID 216) in
3288 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:37 INFO TaskSetManager: Starting task 3.0 in stage 50.0 (TID 224)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:37 INFO TaskSetManager: Finished task 1.0 in stage 47.0 (TID 217) in
3146 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:37 INFO TaskSetManager: Starting task 4.0 in stage 50.0 (TID 225)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:37 INFO TaskSetManager: Finished task 4.0 in stage 45.0 (TID 215) in
3473 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:37 INFO TaskSchedulerImpl: Removed TaskSet 45.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:37 INFO DAGScheduler: ShuffleMapStage 45 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 104.131 s
25/02/06 06:09:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:37 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 146,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 47, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:37 INFO DAGScheduler: failed: Set()
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:37 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:37 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:37 INFO TaskSetManager: Starting task 0.0 in stage 48.0 (TID 226)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:37 INFO TaskSetManager: Finished task 2.0 in stage 47.0 (TID 218) in
3345 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:38 INFO ShufflePartitionsUtil: For shuffle(39, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:38 INFO TaskSetManager: Starting task 1.0 in stage 48.0 (TID 227)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:38 INFO TaskSetManager: Finished task 3.0 in stage 47.0 (TID 219) in
3261 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:38 INFO DAGScheduler: Registering RDD 806 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 134
25/02/06 06:09:38 INFO DAGScheduler: Got map stage job 136 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:38 INFO DAGScheduler: Final stage: ShuffleMapStage 180
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:38 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
179, ShuffleMapStage 97)
25/02/06 06:09:38 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:38 INFO DAGScheduler: Submitting ShuffleMapStage 180
(MapPartitionsRDD[806] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:38 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 180 (MapPartitionsRDD[806] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:38 INFO TaskSchedulerImpl: Adding task set 180.0 with 9 tasks
resource profile 0
25/02/06 06:09:38 INFO TaskSetManager: TaskSet 180.0 using PreferredLocationsV1
25/02/06 06:09:38 INFO FairSchedulableBuilder: Added task set TaskSet_180.0 tasks
to pool 5400065285585087590
25/02/06 06:09:38 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098423) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098423)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:38 INFO TaskSetManager: Starting task 2.0 in stage 48.0 (TID 228)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:38 INFO TaskSetManager: Finished task 4.0 in stage 47.0 (TID 220) in
3341 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:38 INFO TaskSchedulerImpl: Removed TaskSet 47.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:38 INFO DAGScheduler: ShuffleMapStage 47 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 105.563 s
25/02/06 06:09:38 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:38 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 50, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 146,
ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 51, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:38 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:38 INFO DAGScheduler: failed: Set()
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:38 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:38 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:39 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:39 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:39 INFO ShufflePartitionsUtil: For shuffle(53, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:40 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100604) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100604)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:40 INFO DAGScheduler: Registering RDD 814 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 135
25/02/06 06:09:40 INFO DAGScheduler: Got map stage job 137 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:40 INFO DAGScheduler: Final stage: ShuffleMapStage 182
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:40 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 181)
25/02/06 06:09:40 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:40 INFO DAGScheduler: Submitting ShuffleMapStage 182
(MapPartitionsRDD[814] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:40 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 182 (MapPartitionsRDD[814] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:40 INFO TaskSchedulerImpl: Adding task set 182.0 with 9 tasks
resource profile 0
25/02/06 06:09:40 INFO TaskSetManager: TaskSet 182.0 using PreferredLocationsV1
25/02/06 06:09:40 INFO FairSchedulableBuilder: Added task set TaskSet_182.0 tasks
to pool 5400065285585087590
25/02/06 06:09:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:40 INFO TaskSetManager: Starting task 3.0 in stage 48.0 (TID 229)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Finished task 0.0 in stage 50.0 (TID 221) in
3304 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:40 INFO TaskSetManager: Starting task 4.0 in stage 48.0 (TID 230)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Finished task 2.0 in stage 50.0 (TID 223) in
3467 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:40 INFO TaskSetManager: Starting task 0.0 in stage 49.0 (TID 231)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Starting task 1.0 in stage 49.0 (TID 232)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Finished task 4.0 in stage 50.0 (TID 225) in
3618 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:40 INFO TaskSetManager: Finished task 3.0 in stage 50.0 (TID 224) in
3627 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:40 INFO TaskSetManager: Starting task 2.0 in stage 49.0 (TID 233)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Finished task 1.0 in stage 50.0 (TID 222) in
3812 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:40 INFO DAGScheduler: ShuffleMapStage 50 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 107.684 s
25/02/06 06:09:40 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:40 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 48, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 71, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 88,
ShuffleMapStage 67, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 51, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:40 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:40 INFO DAGScheduler: failed: Set()
25/02/06 06:09:40 INFO TaskSchedulerImpl: Removed TaskSet 50.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:40 INFO TaskSetManager: Starting task 3.0 in stage 49.0 (TID 234)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:40 INFO TaskSetManager: Finished task 0.0 in stage 48.0 (TID 226) in
2996 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:41 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:41 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:41 INFO TaskSetManager: Starting task 4.0 in stage 49.0 (TID 235)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:41 INFO TaskSetManager: Finished task 1.0 in stage 48.0 (TID 227) in
3363 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:41 INFO ShufflePartitionsUtil: For shuffle(11, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:42 INFO TaskSetManager: Starting task 0.0 in stage 51.0 (TID 236)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:42 INFO TaskSetManager: Finished task 2.0 in stage 48.0 (TID 228) in
3386 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:42 INFO DAGScheduler: Registering RDD 822 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 136
25/02/06 06:09:42 INFO DAGScheduler: Got map stage job 138 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:42 INFO DAGScheduler: Final stage: ShuffleMapStage 184
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:42 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
183, ShuffleMapStage 97)
25/02/06 06:09:42 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:42 INFO DAGScheduler: Submitting ShuffleMapStage 184
(MapPartitionsRDD[822] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:42 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097564) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097564)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:42 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 184 (MapPartitionsRDD[822] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:42 INFO TaskSchedulerImpl: Adding task set 184.0 with 9 tasks
resource profile 0
25/02/06 06:09:42 INFO TaskSetManager: TaskSet 184.0 using PreferredLocationsV1
25/02/06 06:09:42 INFO FairSchedulableBuilder: Added task set TaskSet_184.0 tasks
to pool 5400065285585087590
25/02/06 06:09:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:43 INFO TaskSetManager: Starting task 1.0 in stage 51.0 (TID 237)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:43 INFO TaskSetManager: Finished task 3.0 in stage 48.0 (TID 229) in
3047 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:44 INFO TaskSetManager: Starting task 2.0 in stage 51.0 (TID 238)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 1.0 in stage 49.0 (TID 232) in
3100 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:44 INFO TaskSetManager: Starting task 3.0 in stage 51.0 (TID 239)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 3.0 in stage 49.0 (TID 234) in
3209 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:44 INFO TaskSetManager: Starting task 4.0 in stage 51.0 (TID 240)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 4.0 in stage 48.0 (TID 230) in
3581 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:44 INFO TaskSchedulerImpl: Removed TaskSet 48.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:44 INFO DAGScheduler: ShuffleMapStage 48 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 111.049 s
25/02/06 06:09:44 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:44 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 182, ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 49, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:44 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:44 INFO DAGScheduler: failed: Set()
25/02/06 06:09:44 INFO TaskSetManager: Starting task 0.0 in stage 52.0 (TID 241)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 2.0 in stage 49.0 (TID 233) in
3388 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:44 INFO MemoryStore: Block broadcast_1 stored as values in memory
(estimated size 183.2 KiB, free 20.2 GiB)
25/02/06 06:09:44 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in
memory (estimated size 59.0 KiB, free 20.2 GiB)
25/02/06 06:09:44 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on
10.183.232.44:37705 (size: 59.0 KiB, free: 20.2 GiB)
25/02/06 06:09:44 INFO SparkContext: Created broadcast 1 from broadcast at
TaskSetManager.scala:711
25/02/06 06:09:44 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on
10.183.229.123:38441 (size: 59.0 KiB, free: 18.4 GiB)
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:44 INFO TaskSetManager: Starting task 1.0 in stage 52.0 (TID 242)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 4.0 in stage 49.0 (TID 235) in
2894 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:44 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on
10.183.232.45:45579 (size: 59.0 KiB, free: 18.4 GiB)
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:44 INFO TaskSetManager: Starting task 2.0 in stage 52.0 (TID 243)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 0.0 in stage 49.0 (TID 231) in
3558 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:44 INFO TaskSchedulerImpl: Removed TaskSet 49.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:44 INFO DAGScheduler: ShuffleMapStage 49 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 111.203 s
25/02/06 06:09:44 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:44 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 52, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 182, ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166,
ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:09:44 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:44 INFO DAGScheduler: failed: Set()
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:44 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:44 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:44 INFO TaskSetManager: Starting task 3.0 in stage 52.0 (TID 244)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:44 INFO TaskSetManager: Finished task 1.0 in stage 52.0 (TID 242) in
559 ms on 10.183.232.45 (executor 2) (1/8)
25/02/06 06:09:44 INFO ShufflePartitionsUtil: For shuffle(22, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:45 INFO TaskSetManager: Starting task 4.0 in stage 52.0 (TID 245)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 0.0 in stage 52.0 (TID 241) in
660 ms on 10.183.229.123 (executor 0) (2/8)
25/02/06 06:09:45 INFO TaskSetManager: Starting task 5.0 in stage 52.0 (TID 246)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 2.0 in stage 52.0 (TID 243) in
561 ms on 10.183.232.45 (executor 2) (3/8)
25/02/06 06:09:45 INFO TaskSetManager: Starting task 6.0 in stage 52.0 (TID 247)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 3.0 in stage 52.0 (TID 244) in
441 ms on 10.183.232.45 (executor 2) (4/8)
25/02/06 06:09:45 INFO TaskSetManager: Finished task 4.0 in stage 52.0 (TID 245) in
397 ms on 10.183.229.123 (executor 0) (5/8)
25/02/06 06:09:45 INFO TaskSetManager: Starting task 7.0 in stage 52.0 (TID 248)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 5.0 in stage 52.0 (TID 246) in
401 ms on 10.183.232.45 (executor 2) (6/8)
25/02/06 06:09:45 INFO TaskSetManager: Starting task 0.0 in stage 53.0 (TID 249)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:45 INFO DAGScheduler: Registering RDD 830 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 137
25/02/06 06:09:45 INFO DAGScheduler: Got map stage job 139 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:45 INFO DAGScheduler: Final stage: ShuffleMapStage 186
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:45 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 185)
25/02/06 06:09:45 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:45 INFO DAGScheduler: Submitting ShuffleMapStage 186
(MapPartitionsRDD[830] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:45 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 186 (MapPartitionsRDD[830] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:45 INFO TaskSchedulerImpl: Adding task set 186.0 with 9 tasks
resource profile 0
25/02/06 06:09:45 INFO TaskSetManager: TaskSet 186.0 using PreferredLocationsV1
25/02/06 06:09:45 INFO FairSchedulableBuilder: Added task set TaskSet_186.0 tasks
to pool 5400065285585087590
25/02/06 06:09:45 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098465) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098465)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:45 INFO TaskSetManager: Starting task 1.0 in stage 53.0 (TID 250)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 0.0 in stage 51.0 (TID 236) in
3477 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:45 INFO TaskSetManager: Starting task 2.0 in stage 53.0 (TID 251)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 7.0 in stage 52.0 (TID 248) in
268 ms on 10.183.229.123 (executor 0) (7/8)
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:45 INFO TaskSetManager: Starting task 3.0 in stage 53.0 (TID 252)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:45 INFO TaskSetManager: Finished task 6.0 in stage 52.0 (TID 247) in
403 ms on 10.183.232.45 (executor 2) (8/8)
25/02/06 06:09:45 INFO TaskSchedulerImpl: Removed TaskSet 52.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:45 INFO DAGScheduler: ShuffleMapStage 52 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 112.452 s
25/02/06 06:09:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:45 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 140,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 182, ShuffleMapStage 51, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166,
ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:09:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:45 INFO DAGScheduler: failed: Set()
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:45 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:45 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:46 INFO BlockManagerInfo: Removed broadcast_1_piece0 on
10.183.232.44:37705 in memory (size: 59.0 KiB, free: 20.2 GiB)
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:46 INFO BlockManagerInfo: Removed broadcast_1_piece0 on
10.183.229.123:38441 in memory (size: 59.0 KiB, free: 18.4 GiB)
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:46 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:46 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:46 INFO BlockManagerInfo: Removed broadcast_1_piece0 on
10.183.232.45:45579 in memory (size: 59.0 KiB, free: 18.4 GiB)
25/02/06 06:09:46 INFO TaskSetManager: Starting task 4.0 in stage 53.0 (TID 253)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:46 INFO TaskSetManager: Finished task 1.0 in stage 51.0 (TID 237) in
3411 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:46 INFO ShufflePartitionsUtil: For shuffle(34, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:47 INFO TaskSetManager: Starting task 0.0 in stage 54.0 (TID 254)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:47 INFO TaskSetManager: Finished task 2.0 in stage 51.0 (TID 238) in
3064 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:47 INFO DAGScheduler: Registering RDD 838 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 138
25/02/06 06:09:47 INFO DAGScheduler: Got map stage job 140 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:47 INFO DAGScheduler: Final stage: ShuffleMapStage 188
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:47 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
187, ShuffleMapStage 97)
25/02/06 06:09:47 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:47 INFO DAGScheduler: Submitting ShuffleMapStage 188
(MapPartitionsRDD[838] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:47 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 188 (MapPartitionsRDD[838] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:47 INFO TaskSchedulerImpl: Adding task set 188.0 with 9 tasks
resource profile 0
25/02/06 06:09:47 INFO TaskSetManager: TaskSet 188.0 using PreferredLocationsV1
25/02/06 06:09:47 INFO FairSchedulableBuilder: Added task set TaskSet_188.0 tasks
to pool 5400065285585087590
25/02/06 06:09:47 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100483) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100483)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
01),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#412),isnotnull(QUANT_B#399),(BILL_DATE#412 = 2022-11-
01),isnotnull(BILL_TYPE#413),isnotnull(MATERIAL#370),NOT (QUANT_B#399 = 0.000),NOT
(BILL_TYPE#413 = ZF8),NOT (BILL_TYPE#413 = ZS1),NOT (BILL_TYPE#413 = F8),NOT
(BILL_TYPE#413 = ZP02),DISTR_CHAN#364 IN (20,25),COMP_CODE#362 INSET 1000, 2000,
3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900,
8000, 8100, 8200,dynamicpruning#308662 308661
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
02),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#1474),isnotnull(QUANT_B#1461),(BILL_DATE#1474 = 2022-11-
02),isnotnull(BILL_TYPE#1475),isnotnull(MATERIAL#1432),NOT (QUANT_B#1461 =
0.000),NOT (BILL_TYPE#1475 = ZF8),NOT (BILL_TYPE#1475 = ZS1),NOT (BILL_TYPE#1475 =
F8),NOT (BILL_TYPE#1475 = ZP02),DISTR_CHAN#1426 IN (20,25),COMP_CODE#1424 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308664 308663
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
03),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#3523),isnotnull(QUANT_B#3510),(BILL_DATE#3523 = 2022-11-
03),isnotnull(BILL_TYPE#3524),isnotnull(MATERIAL#3481),NOT (QUANT_B#3510 =
0.000),NOT (BILL_TYPE#3524 = ZF8),NOT (BILL_TYPE#3524 = ZS1),NOT (BILL_TYPE#3524 =
F8),NOT (BILL_TYPE#3524 = ZP02),DISTR_CHAN#3475 IN (20,25),COMP_CODE#3473 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308666 308665
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
04),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#5994),isnotnull(QUANT_B#5981),(BILL_DATE#5994 = 2022-11-
04),isnotnull(BILL_TYPE#5995),isnotnull(MATERIAL#5952),NOT (QUANT_B#5981 =
0.000),NOT (BILL_TYPE#5995 = ZF8),NOT (BILL_TYPE#5995 = ZS1),NOT (BILL_TYPE#5995 =
F8),NOT (BILL_TYPE#5995 = ZP02),DISTR_CHAN#5946 IN (20,25),COMP_CODE#5944 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308668 308667
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
05),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#8887),isnotnull(QUANT_B#8874),(BILL_DATE#8887 = 2022-11-
05),isnotnull(BILL_TYPE#8888),isnotnull(MATERIAL#8845),NOT (QUANT_B#8874 =
0.000),NOT (BILL_TYPE#8888 = ZF8),NOT (BILL_TYPE#8888 = ZS1),NOT (BILL_TYPE#8888 =
F8),NOT (BILL_TYPE#8888 = ZP02),DISTR_CHAN#8839 IN (20,25),COMP_CODE#8837 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200,dynamicpruning#308670 308669
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
06),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#12202),isnotnull(QUANT_B#12189),(BILL_DATE#12202 = 2022-11-
06),isnotnull(BILL_TYPE#12203),isnotnull(MATERIAL#12160),NOT (QUANT_B#12189 =
0.000),NOT (BILL_TYPE#12203 = ZF8),NOT (BILL_TYPE#12203 = ZS1),NOT (BILL_TYPE#12203
= F8),NOT (BILL_TYPE#12203 = ZP02),DISTR_CHAN#12154 IN (20,25),COMP_CODE#12152
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308672 308671
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
07),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#15939),isnotnull(QUANT_B#15926),(BILL_DATE#15939 = 2022-11-
07),isnotnull(BILL_TYPE#15940),isnotnull(MATERIAL#15897),NOT (QUANT_B#15926 =
0.000),NOT (BILL_TYPE#15940 = ZF8),NOT (BILL_TYPE#15940 = ZS1),NOT (BILL_TYPE#15940
= F8),NOT (BILL_TYPE#15940 = ZP02),DISTR_CHAN#15891 IN (20,25),COMP_CODE#15889
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308674 308673
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
08),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#20098),isnotnull(QUANT_B#20085),(BILL_DATE#20098 = 2022-11-
08),isnotnull(BILL_TYPE#20099),isnotnull(MATERIAL#20056),NOT (QUANT_B#20085 =
0.000),NOT (BILL_TYPE#20099 = ZF8),NOT (BILL_TYPE#20099 = ZS1),NOT (BILL_TYPE#20099
= F8),NOT (BILL_TYPE#20099 = ZP02),DISTR_CHAN#20050 IN (20,25),COMP_CODE#20048
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308676 308675
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
09),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#24679),isnotnull(QUANT_B#24666),(BILL_DATE#24679 = 2022-11-
09),isnotnull(BILL_TYPE#24680),isnotnull(MATERIAL#24637),NOT (QUANT_B#24666 =
0.000),NOT (BILL_TYPE#24680 = ZF8),NOT (BILL_TYPE#24680 = ZS1),NOT (BILL_TYPE#24680
= F8),NOT (BILL_TYPE#24680 = ZP02),DISTR_CHAN#24631 IN (20,25),COMP_CODE#24629
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308678 308677
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
10),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#29682),isnotnull(QUANT_B#29669),(BILL_DATE#29682 = 2022-11-
10),isnotnull(BILL_TYPE#29683),isnotnull(MATERIAL#29640),NOT (QUANT_B#29669 =
0.000),NOT (BILL_TYPE#29683 = ZF8),NOT (BILL_TYPE#29683 = ZS1),NOT (BILL_TYPE#29683
= F8),NOT (BILL_TYPE#29683 = ZP02),DISTR_CHAN#29634 IN (20,25),COMP_CODE#29632
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308680 308679
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
11),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#35107),isnotnull(QUANT_B#35094),(BILL_DATE#35107 = 2022-11-
11),isnotnull(BILL_TYPE#35108),isnotnull(MATERIAL#35065),NOT (QUANT_B#35094 =
0.000),NOT (BILL_TYPE#35108 = ZF8),NOT (BILL_TYPE#35108 = ZS1),NOT (BILL_TYPE#35108
= F8),NOT (BILL_TYPE#35108 = ZP02),DISTR_CHAN#35059 IN (20,25),COMP_CODE#35057
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308682 308681
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
12),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#40954),isnotnull(QUANT_B#40941),(BILL_DATE#40954 = 2022-11-
12),isnotnull(BILL_TYPE#40955),isnotnull(MATERIAL#40912),NOT (QUANT_B#40941 =
0.000),NOT (BILL_TYPE#40955 = ZF8),NOT (BILL_TYPE#40955 = ZS1),NOT (BILL_TYPE#40955
= F8),NOT (BILL_TYPE#40955 = ZP02),DISTR_CHAN#40906 IN (20,25),COMP_CODE#40904
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308684 308683
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
13),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#47223),isnotnull(QUANT_B#47210),(BILL_DATE#47223 = 2022-11-
13),isnotnull(BILL_TYPE#47224),isnotnull(MATERIAL#47181),NOT (QUANT_B#47210 =
0.000),NOT (BILL_TYPE#47224 = ZF8),NOT (BILL_TYPE#47224 = ZS1),NOT (BILL_TYPE#47224
= F8),NOT (BILL_TYPE#47224 = ZP02),DISTR_CHAN#47175 IN (20,25),COMP_CODE#47173
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308686 308685
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
14),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#53914),isnotnull(QUANT_B#53901),(BILL_DATE#53914 = 2022-11-
14),isnotnull(BILL_TYPE#53915),isnotnull(MATERIAL#53872),NOT (QUANT_B#53901 =
0.000),NOT (BILL_TYPE#53915 = ZF8),NOT (BILL_TYPE#53915 = ZS1),NOT (BILL_TYPE#53915
= F8),NOT (BILL_TYPE#53915 = ZP02),DISTR_CHAN#53866 IN (20,25),COMP_CODE#53864
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308688 308687
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
15),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#61027),isnotnull(QUANT_B#61014),(BILL_DATE#61027 = 2022-11-
15),isnotnull(BILL_TYPE#61028),isnotnull(MATERIAL#60985),NOT (QUANT_B#61014 =
0.000),NOT (BILL_TYPE#61028 = ZF8),NOT (BILL_TYPE#61028 = ZS1),NOT (BILL_TYPE#61028
= F8),NOT (BILL_TYPE#61028 = ZP02),DISTR_CHAN#60979 IN (20,25),COMP_CODE#60977
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308690 308689
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
16),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#68562),isnotnull(QUANT_B#68549),(BILL_DATE#68562 = 2022-11-
16),isnotnull(BILL_TYPE#68563),isnotnull(MATERIAL#68520),NOT (QUANT_B#68549 =
0.000),NOT (BILL_TYPE#68563 = ZF8),NOT (BILL_TYPE#68563 = ZS1),NOT (BILL_TYPE#68563
= F8),NOT (BILL_TYPE#68563 = ZP02),DISTR_CHAN#68514 IN (20,25),COMP_CODE#68512
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308692 308691
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
17),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#76519),isnotnull(QUANT_B#76506),(BILL_DATE#76519 = 2022-11-
17),isnotnull(BILL_TYPE#76520),isnotnull(MATERIAL#76477),NOT (QUANT_B#76506 =
0.000),NOT (BILL_TYPE#76520 = ZF8),NOT (BILL_TYPE#76520 = ZS1),NOT (BILL_TYPE#76520
= F8),NOT (BILL_TYPE#76520 = ZP02),DISTR_CHAN#76471 IN (20,25),COMP_CODE#76469
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308694 308693
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
18),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#84898),isnotnull(QUANT_B#84885),(BILL_DATE#84898 = 2022-11-
18),isnotnull(BILL_TYPE#84899),isnotnull(MATERIAL#84856),NOT (QUANT_B#84885 =
0.000),NOT (BILL_TYPE#84899 = ZF8),NOT (BILL_TYPE#84899 = ZS1),NOT (BILL_TYPE#84899
= F8),NOT (BILL_TYPE#84899 = ZP02),DISTR_CHAN#84850 IN (20,25),COMP_CODE#84848
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308696 308695
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
19),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#93634),isnotnull(QUANT_B#93621),(BILL_DATE#93634 = 2022-11-
19),isnotnull(BILL_TYPE#93635),isnotnull(MATERIAL#93592),NOT (QUANT_B#93621 =
0.000),NOT (BILL_TYPE#93635 = ZF8),NOT (BILL_TYPE#93635 = ZS1),NOT (BILL_TYPE#93635
= F8),NOT (BILL_TYPE#93635 = ZP02),DISTR_CHAN#93586 IN (20,25),COMP_CODE#93584
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200,dynamicpruning#308698 308697
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
20),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#102922),isnotnull(QUANT_B#102909),(BILL_DATE#102922 = 2022-11-
20),isnotnull(BILL_TYPE#102923),isnotnull(MATERIAL#102880),NOT (QUANT_B#102909 =
0.000),NOT (BILL_TYPE#102923 = ZF8),NOT (BILL_TYPE#102923 = ZS1),NOT
(BILL_TYPE#102923 = F8),NOT (BILL_TYPE#102923 = ZP02),DISTR_CHAN#102874 IN
(20,25),COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308700 308699
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
21),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#112567),isnotnull(QUANT_B#112554),(BILL_DATE#112567 = 2022-11-
21),isnotnull(BILL_TYPE#112568),isnotnull(MATERIAL#112525),NOT (QUANT_B#112554 =
0.000),NOT (BILL_TYPE#112568 = ZF8),NOT (BILL_TYPE#112568 = ZS1),NOT
(BILL_TYPE#112568 = F8),NOT (BILL_TYPE#112568 = ZP02),DISTR_CHAN#112519 IN
(20,25),COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308702 308701
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
22),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#122634),isnotnull(QUANT_B#122621),(BILL_DATE#122634 = 2022-11-
22),isnotnull(BILL_TYPE#122635),isnotnull(MATERIAL#122592),NOT (QUANT_B#122621 =
0.000),NOT (BILL_TYPE#122635 = ZF8),NOT (BILL_TYPE#122635 = ZS1),NOT
(BILL_TYPE#122635 = F8),NOT (BILL_TYPE#122635 = ZP02),DISTR_CHAN#122586 IN
(20,25),COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308704 308703
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
23),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#133123),isnotnull(QUANT_B#133110),(BILL_DATE#133123 = 2022-11-
23),isnotnull(BILL_TYPE#133124),isnotnull(MATERIAL#133081),NOT (QUANT_B#133110 =
0.000),NOT (BILL_TYPE#133124 = ZF8),NOT (BILL_TYPE#133124 = ZS1),NOT
(BILL_TYPE#133124 = F8),NOT (BILL_TYPE#133124 = ZP02),DISTR_CHAN#133075 IN
(20,25),COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308706 308705
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
24),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#144034),isnotnull(QUANT_B#144021),(BILL_DATE#144034 = 2022-11-
24),isnotnull(BILL_TYPE#144035),isnotnull(MATERIAL#143992),NOT (QUANT_B#144021 =
0.000),NOT (BILL_TYPE#144035 = ZF8),NOT (BILL_TYPE#144035 = ZS1),NOT
(BILL_TYPE#144035 = F8),NOT (BILL_TYPE#144035 = ZP02),DISTR_CHAN#143986 IN
(20,25),COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308708 308707
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
25),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#155367),isnotnull(QUANT_B#155354),(BILL_DATE#155367 = 2022-11-
25),isnotnull(BILL_TYPE#155368),isnotnull(MATERIAL#155325),NOT (QUANT_B#155354 =
0.000),NOT (BILL_TYPE#155368 = ZF8),NOT (BILL_TYPE#155368 = ZS1),NOT
(BILL_TYPE#155368 = F8),NOT (BILL_TYPE#155368 = ZP02),DISTR_CHAN#155319 IN
(20,25),COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308710 308709
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
26),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#167122),isnotnull(QUANT_B#167109),(BILL_DATE#167122 = 2022-11-
26),isnotnull(BILL_TYPE#167123),isnotnull(MATERIAL#167080),NOT (QUANT_B#167109 =
0.000),NOT (BILL_TYPE#167123 = ZF8),NOT (BILL_TYPE#167123 = ZS1),NOT
(BILL_TYPE#167123 = F8),NOT (BILL_TYPE#167123 = ZP02),DISTR_CHAN#167074 IN
(20,25),COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308712 308711
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
27),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#179299),isnotnull(QUANT_B#179286),(BILL_DATE#179299 = 2022-11-
27),isnotnull(BILL_TYPE#179300),isnotnull(MATERIAL#179257),NOT (QUANT_B#179286 =
0.000),NOT (BILL_TYPE#179300 = ZF8),NOT (BILL_TYPE#179300 = ZS1),NOT
(BILL_TYPE#179300 = F8),NOT (BILL_TYPE#179300 = ZP02),DISTR_CHAN#179251 IN
(20,25),COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308714 308713
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
28),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#191898),isnotnull(QUANT_B#191885),(BILL_DATE#191898 = 2022-11-
28),isnotnull(BILL_TYPE#191899),isnotnull(MATERIAL#191856),NOT (QUANT_B#191885 =
0.000),NOT (BILL_TYPE#191899 = ZF8),NOT (BILL_TYPE#191899 = ZS1),NOT
(BILL_TYPE#191899 = F8),NOT (BILL_TYPE#191899 = ZP02),DISTR_CHAN#191850 IN
(20,25),COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308716 308715
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
29),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#204919),isnotnull(QUANT_B#204906),(BILL_DATE#204919 = 2022-11-
29),isnotnull(BILL_TYPE#204920),isnotnull(MATERIAL#204877),NOT (QUANT_B#204906 =
0.000),NOT (BILL_TYPE#204920 = ZF8),NOT (BILL_TYPE#204920 = ZS1),NOT
(BILL_TYPE#204920 = F8),NOT (BILL_TYPE#204920 = ZP02),DISTR_CHAN#204871 IN
(20,25),COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308718 308717
25/02/06 06:09:47 INFO FileSourceStrategy: Pushed Filters:
IsNotNull(BILL_DATE),IsNotNull(QUANT_B),EqualTo(BILL_DATE,2022-11-
30),IsNotNull(BILL_TYPE),IsNotNull(MATERIAL),Not(EqualTo(QUANT_B,0.000)),Not(EqualT
o(BILL_TYPE,ZF8)),Not(EqualTo(BILL_TYPE,ZS1)),Not(EqualTo(BILL_TYPE,F8)),Not(EqualT
o(BILL_TYPE,ZP02)),In(DISTR_CHAN, [20,25]),In(COMP_CODE,
[1000,2000,3000,3200,3400,3600,7000,7200,7400,7450,7500,7550,7600,7700,7800,7900,80
00,8100,8200])
25/02/06 06:09:47 INFO FileSourceStrategy: Post-Scan Filters:
isnotnull(BILL_DATE#218362),isnotnull(QUANT_B#218349),(BILL_DATE#218362 = 2022-11-
30),isnotnull(BILL_TYPE#218363),isnotnull(MATERIAL#218320),NOT (QUANT_B#218349 =
0.000),NOT (BILL_TYPE#218363 = ZF8),NOT (BILL_TYPE#218363 = ZS1),NOT
(BILL_TYPE#218363 = F8),NOT (BILL_TYPE#218363 = ZP02),DISTR_CHAN#218314 IN
(20,25),COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100,
8200,dynamicpruning#308720 308719
25/02/06 06:09:47 INFO TaskSetManager: Starting task 1.0 in stage 54.0 (TID 255)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:47 INFO TaskSetManager: Starting task 2.0 in stage 54.0 (TID 256)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:47 INFO TaskSetManager: Finished task 3.0 in stage 51.0 (TID 239) in
3618 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:47 INFO TaskSetManager: Finished task 4.0 in stage 51.0 (TID 240) in
3518 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:47 INFO TaskSchedulerImpl: Removed TaskSet 51.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:47 INFO DAGScheduler: ShuffleMapStage 51 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 114.496 s
25/02/06 06:09:47 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:47 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ShuffleMapStage 140,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 53, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166,
ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:09:47 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:47 INFO DAGScheduler: failed: Set()
25/02/06 06:09:48 INFO TaskSetManager: Starting task 3.0 in stage 54.0 (TID 257)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:48 INFO TaskSetManager: Finished task 0.0 in stage 53.0 (TID 249) in
3264 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:48 INFO TaskSetManager: Starting task 4.0 in stage 54.0 (TID 258)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:48 INFO TaskSetManager: Finished task 1.0 in stage 53.0 (TID 250) in
3230 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:49 INFO TaskSetManager: Starting task 0.0 in stage 55.0 (TID 259)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:49 INFO TaskSetManager: Finished task 3.0 in stage 53.0 (TID 252) in
3264 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:49 INFO TaskSetManager: Starting task 1.0 in stage 55.0 (TID 260)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:49 INFO TaskSetManager: Finished task 2.0 in stage 53.0 (TID 251) in
3438 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:49 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097731) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097731)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:49 INFO SparkContext: Starting job:
$anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63
25/02/06 06:09:49 INFO DAGScheduler: Got job 141
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:09:49 INFO DAGScheduler: Final stage: ResultStage 190
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:09:49 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
189)
25/02/06 06:09:49 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:49 INFO DAGScheduler: Submitting ResultStage 190
(MapPartitionsRDD[842] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:09:49 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage
190 (MapPartitionsRDD[842] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:09:49 INFO TaskSchedulerImpl: Adding task set 190.0 with 1 tasks
resource profile 0
25/02/06 06:09:49 INFO TaskSetManager: TaskSet 190.0 using PreferredLocationsV1
25/02/06 06:09:49 INFO FairSchedulableBuilder: Added task set TaskSet_190.0 tasks
to pool 5400065285585087590
25/02/06 06:09:49 INFO TaskSetManager: Starting task 2.0 in stage 55.0 (TID 261)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:49 INFO TaskSetManager: Finished task 4.0 in stage 53.0 (TID 253) in
3181 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:49 INFO TaskSchedulerImpl: Removed TaskSet 53.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:49 INFO DAGScheduler: ShuffleMapStage 53 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 116.403 s
25/02/06 06:09:49 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:49 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 54, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 63, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166,
ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:09:49 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:49 INFO DAGScheduler: failed: Set()
25/02/06 06:09:50 INFO ShufflePartitionsUtil: For shuffle(31, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:50 INFO TaskSetManager: Finished task 0.0 in stage 54.0 (TID 254) in
3454 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:50 INFO TaskSetManager: Starting task 3.0 in stage 55.0 (TID 262)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:50 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097510) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097510)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:50 INFO DAGScheduler: Registering RDD 850 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 139
25/02/06 06:09:50 INFO DAGScheduler: Got map stage job 142 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:50 INFO DAGScheduler: Final stage: ShuffleMapStage 192
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
191, ShuffleMapStage 97)
25/02/06 06:09:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:50 INFO DAGScheduler: Submitting ShuffleMapStage 192
(MapPartitionsRDD[850] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:50 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 192 (MapPartitionsRDD[850] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:50 INFO TaskSchedulerImpl: Adding task set 192.0 with 9 tasks
resource profile 0
25/02/06 06:09:50 INFO TaskSetManager: TaskSet 192.0 using PreferredLocationsV1
25/02/06 06:09:50 INFO FairSchedulableBuilder: Added task set TaskSet_192.0 tasks
to pool 5400065285585087590
25/02/06 06:09:51 INFO TaskSetManager: Starting task 4.0 in stage 55.0 (TID 263)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:51 INFO TaskSetManager: Finished task 1.0 in stage 54.0 (TID 255) in
3504 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:51 INFO ShufflePartitionsUtil: For shuffle(27, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:51 INFO TaskSetManager: Finished task 2.0 in stage 54.0 (TID 256) in
3573 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:51 INFO TaskSetManager: Starting task 0.0 in stage 56.0 (TID 264)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:52 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097430) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097430)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:52 INFO DAGScheduler: Registering RDD 858 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 140
25/02/06 06:09:52 INFO DAGScheduler: Got map stage job 143 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:52 INFO DAGScheduler: Final stage: ShuffleMapStage 194
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:52 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 193)
25/02/06 06:09:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:52 INFO DAGScheduler: Submitting ShuffleMapStage 194
(MapPartitionsRDD[858] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:52 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 194 (MapPartitionsRDD[858] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:52 INFO TaskSchedulerImpl: Adding task set 194.0 with 9 tasks
resource profile 0
25/02/06 06:09:52 INFO TaskSetManager: TaskSet 194.0 using PreferredLocationsV1
25/02/06 06:09:52 INFO FairSchedulableBuilder: Added task set TaskSet_194.0 tasks
to pool 5400065285585087590
25/02/06 06:09:52 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:52 INFO TaskSetManager: Starting task 1.0 in stage 56.0 (TID 265)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:52 INFO TaskSetManager: Finished task 3.0 in stage 54.0 (TID 257) in
3578 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:52 INFO TaskSetManager: Starting task 2.0 in stage 56.0 (TID 266)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:52 INFO TaskSetManager: Finished task 0.0 in stage 55.0 (TID 259) in
3567 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:52 INFO TaskSetManager: Starting task 3.0 in stage 56.0 (TID 267)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:52 INFO TaskSetManager: Finished task 4.0 in stage 54.0 (TID 258) in
3800 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:52 INFO TaskSchedulerImpl: Removed TaskSet 54.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:52 INFO DAGScheduler: ShuffleMapStage 54 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 119.312 s
25/02/06 06:09:52 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:52 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 65,
ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 192,
ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 57, ShuffleMapStage 138,
ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 182,
ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74,
ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 55,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180,
ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:52 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:52 INFO DAGScheduler: failed: Set()
25/02/06 06:09:52 INFO TaskSetManager: Starting task 4.0 in stage 56.0 (TID 268)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:52 INFO TaskSetManager: Finished task 1.0 in stage 55.0 (TID 260) in
3819 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:53 INFO ShufflePartitionsUtil: For shuffle(58, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:53 INFO TaskSetManager: Starting task 0.0 in stage 63.0 (TID 269)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:53 INFO TaskSetManager: Finished task 2.0 in stage 55.0 (TID 261) in
3783 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:53 INFO TaskSetManager: Starting task 1.0 in stage 63.0 (TID 270)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:09:53 INFO TaskSetManager: Finished task 3.0 in stage 55.0 (TID 262) in
3263 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:09:54 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097736) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097736)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:54 INFO DAGScheduler: Registering RDD 866 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 141
25/02/06 06:09:54 INFO DAGScheduler: Got map stage job 144 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:54 INFO DAGScheduler: Final stage: ShuffleMapStage 196
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:54 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
195, ShuffleMapStage 97)
25/02/06 06:09:54 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:54 INFO DAGScheduler: Submitting ShuffleMapStage 196
(MapPartitionsRDD[866] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:54 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 196 (MapPartitionsRDD[866] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:54 INFO TaskSchedulerImpl: Adding task set 196.0 with 9 tasks
resource profile 0
25/02/06 06:09:54 INFO TaskSetManager: TaskSet 196.0 using PreferredLocationsV1
25/02/06 06:09:54 INFO FairSchedulableBuilder: Added task set TaskSet_196.0 tasks
to pool 5400065285585087590
25/02/06 06:09:55 INFO TaskSetManager: Starting task 2.0 in stage 63.0 (TID 271)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:55 INFO TaskSetManager: Finished task 0.0 in stage 56.0 (TID 264) in
3629 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:09:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:55 INFO TaskSetManager: Starting task 3.0 in stage 63.0 (TID 272)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:09:55 INFO TaskSetManager: Finished task 4.0 in stage 55.0 (TID 263) in
3848 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:55 INFO TaskSchedulerImpl: Removed TaskSet 55.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:55 INFO DAGScheduler: ShuffleMapStage 55 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 121.774 s
25/02/06 06:09:55 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:55 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 192, ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 57,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 59,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184,
ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180,
ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 166, ShuffleMapStage 56, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:55 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:55 INFO DAGScheduler: failed: Set()
25/02/06 06:09:55 INFO TaskSetManager: Starting task 4.0 in stage 63.0 (TID 273)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:55 INFO TaskSetManager: Finished task 1.0 in stage 56.0 (TID 265) in
3213 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:55 INFO ShufflePartitionsUtil: For shuffle(60, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:56 INFO TaskSetManager: Starting task 0.0 in stage 57.0 (TID 274)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:09:56 INFO TaskSetManager: Finished task 2.0 in stage 56.0 (TID 266) in
3471 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:09:56 INFO DAGScheduler: Registering RDD 874 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 142
25/02/06 06:09:56 INFO DAGScheduler: Got map stage job 145 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:56 INFO DAGScheduler: Final stage: ShuffleMapStage 198
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:56 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
197, ShuffleMapStage 97)
25/02/06 06:09:56 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:56 INFO DAGScheduler: Submitting ShuffleMapStage 198
(MapPartitionsRDD[874] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:56 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 198 (MapPartitionsRDD[874] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:56 INFO TaskSchedulerImpl: Adding task set 198.0 with 9 tasks
resource profile 0
25/02/06 06:09:56 INFO TaskSetManager: TaskSet 198.0 using PreferredLocationsV1
25/02/06 06:09:56 INFO FairSchedulableBuilder: Added task set TaskSet_198.0 tasks
to pool 5400065285585087590
25/02/06 06:09:56 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097635) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097635)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:56 INFO TaskSetManager: Starting task 1.0 in stage 57.0 (TID 275)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:56 INFO TaskSetManager: Finished task 3.0 in stage 56.0 (TID 267) in
3692 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:56 INFO TaskSetManager: Starting task 2.0 in stage 57.0 (TID 276)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:09:56 INFO TaskSetManager: Finished task 4.0 in stage 56.0 (TID 268) in
3699 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:09:56 INFO TaskSchedulerImpl: Removed TaskSet 56.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:56 INFO DAGScheduler: ShuffleMapStage 56 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 123.240 s
25/02/06 06:09:56 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:56 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 61, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 63,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:56 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:56 INFO DAGScheduler: failed: Set()
25/02/06 06:09:57 INFO ShufflePartitionsUtil: For shuffle(59, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:57 INFO TaskSetManager: Starting task 3.0 in stage 57.0 (TID 277)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:57 INFO TaskSetManager: Finished task 0.0 in stage 63.0 (TID 269) in
3717 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:57 INFO TaskSetManager: Starting task 4.0 in stage 57.0 (TID 278)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:09:57 INFO TaskSetManager: Finished task 1.0 in stage 63.0 (TID 270) in
3541 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:09:57 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097466) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097466)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:57 INFO DAGScheduler: Registering RDD 882 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 143
25/02/06 06:09:57 INFO DAGScheduler: Got map stage job 146 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:57 INFO DAGScheduler: Final stage: ShuffleMapStage 200
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:57 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 199)
25/02/06 06:09:57 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:57 INFO DAGScheduler: Submitting ShuffleMapStage 200
(MapPartitionsRDD[882] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:57 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 200 (MapPartitionsRDD[882] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:57 INFO TaskSchedulerImpl: Adding task set 200.0 with 9 tasks
resource profile 0
25/02/06 06:09:57 INFO TaskSetManager: TaskSet 200.0 using PreferredLocationsV1
25/02/06 06:09:57 INFO FairSchedulableBuilder: Added task set TaskSet_200.0 tasks
to pool 5400065285585087590
25/02/06 06:09:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:09:58 INFO TaskSetManager: Starting task 0.0 in stage 61.0 (TID 279)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:09:58 INFO TaskSetManager: Finished task 3.0 in stage 63.0 (TID 272) in
3713 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:09:58 INFO TaskSetManager: Starting task 1.0 in stage 61.0 (TID 280)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:09:58 INFO TaskSetManager: Finished task 2.0 in stage 63.0 (TID 271) in
3860 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:09:58 INFO TaskSetManager: Starting task 2.0 in stage 61.0 (TID 281)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:09:58 INFO TaskSetManager: Finished task 4.0 in stage 63.0 (TID 273) in
3444 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:09:58 INFO TaskSchedulerImpl: Removed TaskSet 63.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:09:58 INFO DAGScheduler: ShuffleMapStage 63 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 125.428 s
25/02/06 06:09:58 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:09:58 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 71, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73,
ShuffleMapStage 196, ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198,
ShuffleMapStage 88, ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 61,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 57, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:09:58 INFO DAGScheduler: waiting: Set()
25/02/06 06:09:58 INFO DAGScheduler: failed: Set()
25/02/06 06:09:59 INFO ShufflePartitionsUtil: For shuffle(61, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:09:59 INFO TaskSetManager: Finished task 0.0 in stage 57.0 (TID 274) in
3369 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:09:59 INFO TaskSetManager: Starting task 3.0 in stage 61.0 (TID 282)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:09:59 INFO TaskSetManager: Starting task 4.0 in stage 61.0 (TID 283)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:09:59 INFO TaskSetManager: Finished task 1.0 in stage 57.0 (TID 275) in
3154 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:09:59 INFO DAGScheduler: Registering RDD 890 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 144
25/02/06 06:09:59 INFO DAGScheduler: Got map stage job 147 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:09:59 INFO DAGScheduler: Final stage: ShuffleMapStage 202
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:09:59 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
201, ShuffleMapStage 97)
25/02/06 06:09:59 INFO DAGScheduler: Missing parents: List()
25/02/06 06:09:59 INFO DAGScheduler: Submitting ShuffleMapStage 202
(MapPartitionsRDD[890] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:09:59 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099589) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099589)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:09:59 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 202 (MapPartitionsRDD[890] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:09:59 INFO TaskSchedulerImpl: Adding task set 202.0 with 9 tasks
resource profile 0
25/02/06 06:09:59 INFO TaskSetManager: TaskSet 202.0 using PreferredLocationsV1
25/02/06 06:09:59 INFO FairSchedulableBuilder: Added task set TaskSet_202.0 tasks
to pool 5400065285585087590
25/02/06 06:10:00 INFO TaskSetManager: Starting task 0.0 in stage 58.0 (TID 284)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:00 INFO TaskSetManager: Finished task 2.0 in stage 57.0 (TID 276) in
3491 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:00 INFO DBCEventLoggingListener: Rolling event log;
numTimesRolledOver = 2
25/02/06 06:10:00 INFO DBCEventLoggingListener: Rolled active log file
/databricks/driver/eventlogs/9206972423021441211/eventlog to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-10, size =
17881747
25/02/06 06:10:00 INFO DBCEventLoggingListener: Logging events to
eventlogs/9206972423021441211/eventlog
25/02/06 06:10:00 INFO TaskSetManager: Starting task 1.0 in stage 58.0 (TID 285)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:00 INFO TaskSetManager: Finished task 3.0 in stage 57.0 (TID 277) in
3045 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:00 INFO TaskSetManager: Starting task 2.0 in stage 58.0 (TID 286)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:00 INFO TaskSetManager: Finished task 4.0 in stage 57.0 (TID 278) in
3147 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:00 INFO TaskSchedulerImpl: Removed TaskSet 57.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:00 INFO DAGScheduler: ShuffleMapStage 57 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 127.096 s
25/02/06 06:10:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:00 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 71,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94,
ShuffleMapStage 73, ShuffleMapStage 196, ShuffleMapStage 65, ShuffleMapStage 146,
ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69, ShuffleMapStage 192,
ShuffleMapStage 61, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:00 INFO DAGScheduler: failed: Set()
25/02/06 06:10:00 INFO DBCEventLoggingListener: Compressed rolled file
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-10 to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-10.gz in
478ms, size = 2704130
25/02/06 06:10:00 INFO DBCEventLoggingListener: Deleted rolled file
eventlogs/9206972423021441211/eventlog-2025-02-06--06-10
25/02/06 06:10:00 INFO ShufflePartitionsUtil: For shuffle(56, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:01 INFO DAGScheduler: Registering RDD 898 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 145
25/02/06 06:10:01 INFO DAGScheduler: Got map stage job 148 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:01 INFO DAGScheduler: Final stage: ShuffleMapStage 204
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:01 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
203, ShuffleMapStage 97)
25/02/06 06:10:01 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:01 INFO DAGScheduler: Submitting ShuffleMapStage 204
(MapPartitionsRDD[898] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:01 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097544) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097544)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:01 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 204 (MapPartitionsRDD[898] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:01 INFO TaskSchedulerImpl: Adding task set 204.0 with 9 tasks
resource profile 0
25/02/06 06:10:01 INFO TaskSetManager: TaskSet 204.0 using PreferredLocationsV1
25/02/06 06:10:01 INFO FairSchedulableBuilder: Added task set TaskSet_204.0 tasks
to pool 5400065285585087590
25/02/06 06:10:02 INFO TaskSetManager: Starting task 3.0 in stage 58.0 (TID 287)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:02 INFO TaskSetManager: Finished task 1.0 in stage 61.0 (TID 280) in
3690 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:02 INFO TaskSetManager: Starting task 4.0 in stage 58.0 (TID 288)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:02 INFO TaskSetManager: Finished task 2.0 in stage 61.0 (TID 281) in
3643 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:02 INFO TaskSetManager: Starting task 0.0 in stage 66.0 (TID 289)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:02 INFO TaskSetManager: Finished task 0.0 in stage 61.0 (TID 279) in
3736 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:03 INFO TaskSetManager: Starting task 1.0 in stage 66.0 (TID 290)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:03 INFO TaskSetManager: Finished task 3.0 in stage 61.0 (TID 282) in
3546 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:03 INFO TaskSetManager: Starting task 2.0 in stage 66.0 (TID 291)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:03 INFO TaskSetManager: Finished task 4.0 in stage 61.0 (TID 283) in
3759 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:03 INFO TaskSchedulerImpl: Removed TaskSet 61.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:03 INFO DAGScheduler: ShuffleMapStage 61 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 129.794 s
25/02/06 06:10:03 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:03 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 75, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202,
ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196, ShuffleMapStage 65,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 58, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:03 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:03 INFO DAGScheduler: failed: Set()
25/02/06 06:10:03 INFO TaskSetManager: Starting task 3.0 in stage 66.0 (TID 292)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:03 INFO TaskSetManager: Finished task 0.0 in stage 58.0 (TID 284) in
3731 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:03 INFO ShufflePartitionsUtil: For shuffle(57, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:03 INFO TaskSetManager: Starting task 4.0 in stage 66.0 (TID 293)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:03 INFO TaskSetManager: Finished task 1.0 in stage 58.0 (TID 285) in
3649 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:04 INFO TaskSetManager: Starting task 0.0 in stage 59.0 (TID 294)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:04 INFO TaskSetManager: Finished task 2.0 in stage 58.0 (TID 286) in
3523 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:04 INFO DAGScheduler: Registering RDD 906 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 146
25/02/06 06:10:04 INFO DAGScheduler: Got map stage job 149 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:04 INFO DAGScheduler: Final stage: ShuffleMapStage 206
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
205, ShuffleMapStage 97)
25/02/06 06:10:04 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:04 INFO DAGScheduler: Submitting ShuffleMapStage 206
(MapPartitionsRDD[906] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:04 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099500) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099500)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:04 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 206 (MapPartitionsRDD[906] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:04 INFO TaskSchedulerImpl: Adding task set 206.0 with 9 tasks
resource profile 0
25/02/06 06:10:04 INFO TaskSetManager: TaskSet 206.0 using PreferredLocationsV1
25/02/06 06:10:04 INFO FairSchedulableBuilder: Added task set TaskSet_206.0 tasks
to pool 5400065285585087590
25/02/06 06:10:05 INFO TaskSetManager: Starting task 1.0 in stage 59.0 (TID 295)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:05 INFO TaskSetManager: Finished task 4.0 in stage 58.0 (TID 288) in
3386 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:06 INFO TaskSetManager: Starting task 2.0 in stage 59.0 (TID 296)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:06 INFO TaskSetManager: Finished task 0.0 in stage 66.0 (TID 289) in
3465 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:06 INFO TaskSetManager: Starting task 3.0 in stage 59.0 (TID 297)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:06 INFO TaskSetManager: Finished task 1.0 in stage 66.0 (TID 290) in
3232 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:06 INFO TaskSetManager: Starting task 4.0 in stage 59.0 (TID 298)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:06 INFO TaskSetManager: Finished task 3.0 in stage 58.0 (TID 287) in
3739 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:06 INFO TaskSchedulerImpl: Removed TaskSet 58.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:06 INFO DAGScheduler: ShuffleMapStage 58 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 132.880 s
25/02/06 06:10:06 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:06 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 66,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114,
ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:06 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:06 INFO DAGScheduler: failed: Set()
25/02/06 06:10:06 INFO TaskSetManager: Finished task 2.0 in stage 66.0 (TID 291) in
3313 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:06 INFO TaskSetManager: Starting task 0.0 in stage 60.0 (TID 299)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:06 INFO ShufflePartitionsUtil: For shuffle(55, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:07 INFO TaskSetManager: Starting task 1.0 in stage 60.0 (TID 300)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:07 INFO TaskSetManager: Finished task 3.0 in stage 66.0 (TID 292) in
3445 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:07 INFO DAGScheduler: Registering RDD 914 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 147
25/02/06 06:10:07 INFO DAGScheduler: Got map stage job 150 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:07 INFO DAGScheduler: Final stage: ShuffleMapStage 208
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:07 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
207, ShuffleMapStage 97)
25/02/06 06:10:07 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:07 INFO DAGScheduler: Submitting ShuffleMapStage 208
(MapPartitionsRDD[914] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:07 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 208 (MapPartitionsRDD[914] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:07 INFO TaskSchedulerImpl: Adding task set 208.0 with 9 tasks
resource profile 0
25/02/06 06:10:07 INFO TaskSetManager: TaskSet 208.0 using PreferredLocationsV1
25/02/06 06:10:07 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097477) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097477)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:07 INFO FairSchedulableBuilder: Added task set TaskSet_208.0 tasks
to pool 5400065285585087590
25/02/06 06:10:07 INFO TaskSetManager: Starting task 2.0 in stage 60.0 (TID 301)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:07 INFO TaskSetManager: Finished task 4.0 in stage 66.0 (TID 293) in
3454 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:07 INFO TaskSchedulerImpl: Removed TaskSet 66.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:07 INFO DAGScheduler: ShuffleMapStage 66 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 133.874 s
25/02/06 06:10:07 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:07 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 59, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 62, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85,
ShuffleMapStage 64, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:07 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:07 INFO DAGScheduler: failed: Set()
25/02/06 06:10:07 INFO TaskSetManager: Starting task 3.0 in stage 60.0 (TID 302)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:07 INFO TaskSetManager: Finished task 0.0 in stage 59.0 (TID 294) in
3425 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:07 INFO ShufflePartitionsUtil: For shuffle(64, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:08 INFO DAGScheduler: Registering RDD 922 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 148
25/02/06 06:10:08 INFO DAGScheduler: Got map stage job 151 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:08 INFO DAGScheduler: Final stage: ShuffleMapStage 210
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:08 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
209, ShuffleMapStage 97)
25/02/06 06:10:08 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:08 INFO DAGScheduler: Submitting ShuffleMapStage 210
(MapPartitionsRDD[922] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:08 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 210 (MapPartitionsRDD[922] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:08 INFO TaskSchedulerImpl: Adding task set 210.0 with 9 tasks
resource profile 0
25/02/06 06:10:08 INFO TaskSetManager: TaskSet 210.0 using PreferredLocationsV1
25/02/06 06:10:08 INFO FairSchedulableBuilder: Added task set TaskSet_210.0 tasks
to pool 5400065285585087590
25/02/06 06:10:08 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099655) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099655)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:09 INFO TaskSetManager: Starting task 4.0 in stage 60.0 (TID 303)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:09 INFO TaskSetManager: Finished task 1.0 in stage 59.0 (TID 295) in
3647 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:09 INFO TaskSetManager: Starting task 0.0 in stage 62.0 (TID 304)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:09 INFO TaskSetManager: Finished task 2.0 in stage 59.0 (TID 296) in
3631 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:09 INFO TaskSetManager: Starting task 1.0 in stage 62.0 (TID 305)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:09 INFO TaskSetManager: Finished task 3.0 in stage 59.0 (TID 297) in
3525 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:10 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:10 INFO TaskSetManager: Starting task 2.0 in stage 62.0 (TID 306)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:10 INFO TaskSetManager: Finished task 0.0 in stage 60.0 (TID 299) in
3594 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:10 INFO TaskSetManager: Starting task 3.0 in stage 62.0 (TID 307)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:10 INFO TaskSetManager: Finished task 4.0 in stage 59.0 (TID 298) in
3969 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:10 INFO TaskSchedulerImpl: Removed TaskSet 59.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:10 INFO DAGScheduler: ShuffleMapStage 59 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 136.781 s
25/02/06 06:10:10 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:10 INFO DAGScheduler: running: Set(ShuffleMapStage 102,
ShuffleMapStage 81, ShuffleMapStage 60, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:10 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:10 INFO DAGScheduler: failed: Set()
25/02/06 06:10:10 INFO ShufflePartitionsUtil: For shuffle(65, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:10 INFO TaskSetManager: Starting task 4.0 in stage 62.0 (TID 308)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:10 INFO TaskSetManager: Finished task 2.0 in stage 60.0 (TID 301) in
3359 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:10 INFO TaskSetManager: Starting task 0.0 in stage 65.0 (TID 309)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:10 INFO TaskSetManager: Finished task 3.0 in stage 60.0 (TID 302) in
3435 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:11 INFO TaskSetManager: Starting task 1.0 in stage 65.0 (TID 310)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:11 INFO TaskSetManager: Finished task 1.0 in stage 60.0 (TID 300) in
3752 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:11 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100569) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100569)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:11 INFO DAGScheduler: Registering RDD 930 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 149
25/02/06 06:10:11 INFO DAGScheduler: Got map stage job 152 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:11 INFO DAGScheduler: Final stage: ShuffleMapStage 212
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:11 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 211)
25/02/06 06:10:11 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:11 INFO DAGScheduler: Submitting ShuffleMapStage 212
(MapPartitionsRDD[930] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:11 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 212 (MapPartitionsRDD[930] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:11 INFO TaskSchedulerImpl: Adding task set 212.0 with 9 tasks
resource profile 0
25/02/06 06:10:11 INFO TaskSetManager: TaskSet 212.0 using PreferredLocationsV1
25/02/06 06:10:11 INFO FairSchedulableBuilder: Added task set TaskSet_212.0 tasks
to pool 5400065285585087590
25/02/06 06:10:13 INFO TaskSetManager: Starting task 2.0 in stage 65.0 (TID 311)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:13 INFO TaskSetManager: Finished task 1.0 in stage 62.0 (TID 305) in
3319 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:13 INFO TaskSetManager: Starting task 3.0 in stage 65.0 (TID 312)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:13 INFO TaskSetManager: Finished task 4.0 in stage 60.0 (TID 303) in
3539 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:13 INFO TaskSchedulerImpl: Removed TaskSet 60.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:13 INFO DAGScheduler: ShuffleMapStage 60 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 139.665 s
25/02/06 06:10:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:13 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 62,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:13 INFO DAGScheduler: failed: Set()
25/02/06 06:10:13 INFO TaskSetManager: Starting task 4.0 in stage 65.0 (TID 313)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:13 INFO TaskSetManager: Finished task 0.0 in stage 62.0 (TID 304) in
3607 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:13 INFO TaskSetManager: Finished task 3.0 in stage 62.0 (TID 307) in
3136 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:13 INFO TaskSetManager: Finished task 2.0 in stage 62.0 (TID 306) in
3202 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:13 INFO TaskSetManager: Starting task 0.0 in stage 64.0 (TID 314)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:13 INFO TaskSetManager: Starting task 1.0 in stage 64.0 (TID 315)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:13 INFO ShufflePartitionsUtil: For shuffle(66, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:14 INFO TaskSetManager: Starting task 2.0 in stage 64.0 (TID 316)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:14 INFO TaskSetManager: Finished task 4.0 in stage 62.0 (TID 308) in
3325 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:14 INFO TaskSchedulerImpl: Removed TaskSet 62.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:14 INFO DAGScheduler: ShuffleMapStage 62 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 140.592 s
25/02/06 06:10:14 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:14 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 65, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ShuffleMapStage 67, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 69, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78,
ShuffleMapStage 180, ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 93, ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 85, ShuffleMapStage 64, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:10:14 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:14 INFO DAGScheduler: failed: Set()
25/02/06 06:10:14 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099588) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099588)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:14 INFO TaskSetManager: Starting task 3.0 in stage 64.0 (TID 317)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:14 INFO DAGScheduler: Registering RDD 938 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 150
25/02/06 06:10:14 INFO DAGScheduler: Got map stage job 153 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:14 INFO TaskSetManager: Finished task 0.0 in stage 65.0 (TID 309) in
3560 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:14 INFO DAGScheduler: Final stage: ShuffleMapStage 214
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:14 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
213, ShuffleMapStage 97)
25/02/06 06:10:14 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:14 INFO DAGScheduler: Submitting ShuffleMapStage 214
(MapPartitionsRDD[938] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:14 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 214 (MapPartitionsRDD[938] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:14 INFO TaskSchedulerImpl: Adding task set 214.0 with 9 tasks
resource profile 0
25/02/06 06:10:14 INFO TaskSetManager: TaskSet 214.0 using PreferredLocationsV1
25/02/06 06:10:14 INFO FairSchedulableBuilder: Added task set TaskSet_214.0 tasks
to pool 5400065285585087590
25/02/06 06:10:14 INFO TaskSetManager: Starting task 4.0 in stage 64.0 (TID 318)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:14 INFO TaskSetManager: Finished task 1.0 in stage 65.0 (TID 310) in
3691 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:14 INFO ShufflePartitionsUtil: For shuffle(62, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:15 INFO DAGScheduler: Registering RDD 946 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 151
25/02/06 06:10:15 INFO DAGScheduler: Got map stage job 154 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:15 INFO DAGScheduler: Final stage: ShuffleMapStage 216
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:15 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
215, ShuffleMapStage 97)
25/02/06 06:10:15 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:15 INFO DAGScheduler: Submitting ShuffleMapStage 216
(MapPartitionsRDD[946] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:15 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 216 (MapPartitionsRDD[946] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:15 INFO TaskSchedulerImpl: Adding task set 216.0 with 9 tasks
resource profile 0
25/02/06 06:10:15 INFO TaskSetManager: TaskSet 216.0 using PreferredLocationsV1
25/02/06 06:10:15 INFO FairSchedulableBuilder: Added task set TaskSet_216.0 tasks
to pool 5400065285585087590
25/02/06 06:10:15 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098638) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098638)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:16 INFO TaskSetManager: Starting task 0.0 in stage 69.0 (TID 319)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:16 INFO TaskSetManager: Finished task 2.0 in stage 65.0 (TID 311) in
3307 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:16 INFO TaskSetManager: Starting task 1.0 in stage 69.0 (TID 320)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:16 INFO TaskSetManager: Finished task 3.0 in stage 65.0 (TID 312) in
3518 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:16 INFO TaskSetManager: Starting task 2.0 in stage 69.0 (TID 321)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:16 INFO TaskSetManager: Finished task 0.0 in stage 64.0 (TID 314) in
3412 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:17 INFO TaskSetManager: Starting task 3.0 in stage 69.0 (TID 322)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:17 INFO TaskSetManager: Finished task 4.0 in stage 65.0 (TID 313) in
3750 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:17 INFO TaskSchedulerImpl: Removed TaskSet 65.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:17 INFO DAGScheduler: ShuffleMapStage 65 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 143.516 s
25/02/06 06:10:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:17 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184,
ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180,
ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85, ShuffleMapStage 64,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:17 INFO DAGScheduler: failed: Set()
25/02/06 06:10:17 INFO TaskSetManager: Starting task 4.0 in stage 69.0 (TID 323)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:17 INFO TaskSetManager: Finished task 1.0 in stage 64.0 (TID 315) in
3682 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:17 INFO TaskSetManager: Starting task 0.0 in stage 72.0 (TID 324)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:17 INFO TaskSetManager: Finished task 2.0 in stage 64.0 (TID 316) in
3287 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:17 INFO ShufflePartitionsUtil: For shuffle(63, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:17 INFO TaskSetManager: Starting task 1.0 in stage 72.0 (TID 325)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:17 INFO TaskSetManager: Finished task 3.0 in stage 64.0 (TID 317) in
3505 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:17 INFO DAGScheduler: Registering RDD 954 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 152
25/02/06 06:10:17 INFO DAGScheduler: Got map stage job 155 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:17 INFO DAGScheduler: Final stage: ShuffleMapStage 218
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:17 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 217)
25/02/06 06:10:17 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:17 INFO DAGScheduler: Submitting ShuffleMapStage 218
(MapPartitionsRDD[954] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:17 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 218 (MapPartitionsRDD[954] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:17 INFO TaskSchedulerImpl: Adding task set 218.0 with 9 tasks
resource profile 0
25/02/06 06:10:17 INFO TaskSetManager: TaskSet 218.0 using PreferredLocationsV1
25/02/06 06:10:17 INFO FairSchedulableBuilder: Added task set TaskSet_218.0 tasks
to pool 5400065285585087590
25/02/06 06:10:17 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099546) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099546)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:18 INFO TaskSetManager: Starting task 2.0 in stage 72.0 (TID 326)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:18 INFO TaskSetManager: Finished task 4.0 in stage 64.0 (TID 318) in
3750 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:18 INFO TaskSchedulerImpl: Removed TaskSet 64.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:18 INFO DAGScheduler: ShuffleMapStage 64 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 144.964 s
25/02/06 06:10:18 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:18 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 69,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184,
ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180,
ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 72, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:18 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:18 INFO DAGScheduler: failed: Set()
25/02/06 06:10:18 INFO ShufflePartitionsUtil: For shuffle(67, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:19 INFO DAGScheduler: Registering RDD 962 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 153
25/02/06 06:10:19 INFO DAGScheduler: Got map stage job 156 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:19 INFO DAGScheduler: Final stage: ShuffleMapStage 220
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:19 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
219, ShuffleMapStage 97)
25/02/06 06:10:19 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:19 INFO DAGScheduler: Submitting ShuffleMapStage 220
(MapPartitionsRDD[962] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:19 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 220 (MapPartitionsRDD[962] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:19 INFO TaskSchedulerImpl: Adding task set 220.0 with 9 tasks
resource profile 0
25/02/06 06:10:19 INFO TaskSetManager: TaskSet 220.0 using PreferredLocationsV1
25/02/06 06:10:19 INFO FairSchedulableBuilder: Added task set TaskSet_220.0 tasks
to pool 5400065285585087590
25/02/06 06:10:19 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099528) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099528)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:19 INFO TaskSetManager: Starting task 3.0 in stage 72.0 (TID 327)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:19 INFO TaskSetManager: Finished task 0.0 in stage 69.0 (TID 319) in
3523 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:20 INFO TaskSetManager: Starting task 4.0 in stage 72.0 (TID 328)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:20 INFO TaskSetManager: Finished task 1.0 in stage 69.0 (TID 320) in
3658 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:20 INFO TaskSetManager: Starting task 0.0 in stage 67.0 (TID 329)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:20 INFO TaskSetManager: Finished task 2.0 in stage 69.0 (TID 321) in
3564 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:20 INFO TaskSetManager: Starting task 1.0 in stage 67.0 (TID 330)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:20 INFO TaskSetManager: Finished task 3.0 in stage 69.0 (TID 322) in
3590 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:20 INFO TaskSetManager: Starting task 2.0 in stage 67.0 (TID 331)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:20 INFO TaskSetManager: Finished task 4.0 in stage 69.0 (TID 323) in
3744 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:20 INFO TaskSchedulerImpl: Removed TaskSet 69.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:20 INFO DAGScheduler: ShuffleMapStage 69 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 147.257 s
25/02/06 06:10:20 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:20 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 182,
ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74,
ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 72,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:20 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:20 INFO DAGScheduler: failed: Set()
25/02/06 06:10:20 INFO TaskSetManager: Starting task 3.0 in stage 67.0 (TID 332)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:20 INFO TaskSetManager: Finished task 0.0 in stage 72.0 (TID 324) in
3450 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:21 INFO ShufflePartitionsUtil: For shuffle(68, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:21 INFO TaskSetManager: Starting task 4.0 in stage 67.0 (TID 333)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:21 INFO TaskSetManager: Finished task 1.0 in stage 72.0 (TID 325) in
3332 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:21 INFO DAGScheduler: Registering RDD 970 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 154
25/02/06 06:10:21 INFO DAGScheduler: Got map stage job 157 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:21 INFO DAGScheduler: Final stage: ShuffleMapStage 222
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:21 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
221, ShuffleMapStage 97)
25/02/06 06:10:21 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:21 INFO DAGScheduler: Submitting ShuffleMapStage 222
(MapPartitionsRDD[970] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:21 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 222 (MapPartitionsRDD[970] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:21 INFO TaskSchedulerImpl: Adding task set 222.0 with 9 tasks
resource profile 0
25/02/06 06:10:21 INFO TaskSetManager: TaskSet 222.0 using PreferredLocationsV1
25/02/06 06:10:21 INFO FairSchedulableBuilder: Added task set TaskSet_222.0 tasks
to pool 5400065285585087590
25/02/06 06:10:21 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099558) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099558)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:22 INFO TaskSetManager: Starting task 0.0 in stage 68.0 (TID 334)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:22 INFO TaskSetManager: Finished task 2.0 in stage 72.0 (TID 326) in
3690 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:23 INFO TaskSetManager: Starting task 1.0 in stage 68.0 (TID 335)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:23 INFO TaskSetManager: Finished task 3.0 in stage 72.0 (TID 327) in
3327 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:10:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:10:23 INFO TaskSetManager: Starting task 2.0 in stage 68.0 (TID 336)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:23 INFO TaskSetManager: Finished task 4.0 in stage 72.0 (TID 328) in
3561 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:23 INFO TaskSchedulerImpl: Removed TaskSet 72.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:23 INFO DAGScheduler: ShuffleMapStage 72 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 150.283 s
25/02/06 06:10:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:23 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ShuffleMapStage 67,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 182,
ShuffleMapStage 132, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74,
ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 70, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 89, ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:23 INFO DAGScheduler: failed: Set()
25/02/06 06:10:24 INFO TaskSetManager: Starting task 3.0 in stage 68.0 (TID 337)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:24 INFO TaskSetManager: Finished task 0.0 in stage 67.0 (TID 329) in
3650 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:24 INFO TaskSetManager: Starting task 4.0 in stage 68.0 (TID 338)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:24 INFO TaskSetManager: Finished task 2.0 in stage 67.0 (TID 331) in
3428 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:24 INFO TaskSetManager: Starting task 0.0 in stage 70.0 (TID 339)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:24 INFO TaskSetManager: Finished task 3.0 in stage 67.0 (TID 332) in
3431 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:24 INFO ShufflePartitionsUtil: For shuffle(72, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:24 INFO TaskSetManager: Starting task 1.0 in stage 70.0 (TID 340)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:24 INFO TaskSetManager: Finished task 1.0 in stage 67.0 (TID 330) in
3677 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:24 INFO TaskSetManager: Starting task 2.0 in stage 70.0 (TID 341)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:24 INFO TaskSetManager: Finished task 4.0 in stage 67.0 (TID 333) in
3565 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:24 INFO TaskSchedulerImpl: Removed TaskSet 67.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:24 INFO DAGScheduler: ShuffleMapStage 67 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 151.282 s
25/02/06 06:10:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:24 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 93, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 68, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:10:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:24 INFO DAGScheduler: failed: Set()
25/02/06 06:10:24 INFO DAGScheduler: Registering RDD 978 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 155
25/02/06 06:10:24 INFO DAGScheduler: Got map stage job 158 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:24 INFO DAGScheduler: Final stage: ShuffleMapStage 224
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
223, ShuffleMapStage 97)
25/02/06 06:10:24 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:24 INFO DAGScheduler: Submitting ShuffleMapStage 224
(MapPartitionsRDD[978] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:24 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097627) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097627)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:24 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 224 (MapPartitionsRDD[978] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:24 INFO TaskSchedulerImpl: Adding task set 224.0 with 9 tasks
resource profile 0
25/02/06 06:10:24 INFO TaskSetManager: TaskSet 224.0 using PreferredLocationsV1
25/02/06 06:10:24 INFO FairSchedulableBuilder: Added task set TaskSet_224.0 tasks
to pool 5400065285585087590
25/02/06 06:10:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:25 INFO ShufflePartitionsUtil: For shuffle(71, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:25 INFO TaskSetManager: Starting task 3.0 in stage 70.0 (TID 342)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:25 INFO TaskSetManager: Finished task 0.0 in stage 68.0 (TID 334) in
3380 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:25 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098583) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098583)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:25 INFO DAGScheduler: Registering RDD 986 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 156
25/02/06 06:10:25 INFO DAGScheduler: Got map stage job 159 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:25 INFO DAGScheduler: Final stage: ShuffleMapStage 226
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
225, ShuffleMapStage 97)
25/02/06 06:10:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:25 INFO DAGScheduler: Submitting ShuffleMapStage 226
(MapPartitionsRDD[986] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:25 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 226 (MapPartitionsRDD[986] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:25 INFO TaskSchedulerImpl: Adding task set 226.0 with 9 tasks
resource profile 0
25/02/06 06:10:25 INFO TaskSetManager: TaskSet 226.0 using PreferredLocationsV1
25/02/06 06:10:25 INFO FairSchedulableBuilder: Added task set TaskSet_226.0 tasks
to pool 5400065285585087590
25/02/06 06:10:26 INFO TaskSetManager: Starting task 4.0 in stage 70.0 (TID 343)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:26 INFO TaskSetManager: Finished task 1.0 in stage 68.0 (TID 335) in
3365 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:26 INFO TaskSetManager: Starting task 0.0 in stage 71.0 (TID 344)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:26 INFO TaskSetManager: Finished task 2.0 in stage 68.0 (TID 336) in
3117 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:27 INFO TaskSetManager: Starting task 1.0 in stage 71.0 (TID 345)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:27 INFO TaskSetManager: Finished task 3.0 in stage 68.0 (TID 337) in
3282 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:27 INFO TaskSetManager: Starting task 2.0 in stage 71.0 (TID 346)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:27 INFO TaskSetManager: Finished task 0.0 in stage 70.0 (TID 339) in
3274 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:27 INFO TaskSetManager: Starting task 3.0 in stage 71.0 (TID 347)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:27 INFO TaskSetManager: Finished task 4.0 in stage 68.0 (TID 338) in
3505 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:27 INFO TaskSchedulerImpl: Removed TaskSet 68.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:27 INFO DAGScheduler: ShuffleMapStage 68 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 154.196 s
25/02/06 06:10:27 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:27 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 70,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:27 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:27 INFO DAGScheduler: failed: Set()
25/02/06 06:10:28 INFO TaskSetManager: Starting task 4.0 in stage 71.0 (TID 348)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:28 INFO TaskSetManager: Finished task 1.0 in stage 70.0 (TID 340) in
3697 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:28 INFO ShufflePartitionsUtil: For shuffle(69, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:28 INFO TaskSetManager: Starting task 0.0 in stage 73.0 (TID 349)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:28 INFO TaskSetManager: Finished task 2.0 in stage 70.0 (TID 341) in
3740 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:28 INFO DAGScheduler: Registering RDD 994 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 157
25/02/06 06:10:28 INFO DAGScheduler: Got map stage job 160 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:28 INFO DAGScheduler: Final stage: ShuffleMapStage 228
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:28 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
227, ShuffleMapStage 97)
25/02/06 06:10:28 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:28 INFO DAGScheduler: Submitting ShuffleMapStage 228
(MapPartitionsRDD[994] at mapPartitionsInternal at PhotonExec.scala:483), which has
no missing parents
25/02/06 06:10:28 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 228 (MapPartitionsRDD[994] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:28 INFO TaskSchedulerImpl: Adding task set 228.0 with 9 tasks
resource profile 0
25/02/06 06:10:28 INFO TaskSetManager: TaskSet 228.0 using PreferredLocationsV1
25/02/06 06:10:28 INFO FairSchedulableBuilder: Added task set TaskSet_228.0 tasks
to pool 5400065285585087590
25/02/06 06:10:28 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097621) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097621)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:29 INFO TaskSetManager: Starting task 1.0 in stage 73.0 (TID 350)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:29 INFO TaskSetManager: Finished task 3.0 in stage 70.0 (TID 342) in
3514 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:30 INFO TaskSetManager: Starting task 2.0 in stage 73.0 (TID 351)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:30 INFO TaskSetManager: Finished task 0.0 in stage 71.0 (TID 344) in
3171 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:30 INFO TaskSetManager: Finished task 4.0 in stage 70.0 (TID 343) in
3703 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:30 INFO TaskSchedulerImpl: Removed TaskSet 70.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:30 INFO TaskSetManager: Starting task 3.0 in stage 73.0 (TID 352)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:30 INFO DAGScheduler: ShuffleMapStage 70 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 156.728 s
25/02/06 06:10:30 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:30 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 71, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 130, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 76, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:30 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:30 INFO DAGScheduler: failed: Set()
25/02/06 06:10:30 INFO TaskSetManager: Starting task 4.0 in stage 73.0 (TID 353)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:30 INFO TaskSetManager: Starting task 0.0 in stage 74.0 (TID 354)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:30 INFO TaskSetManager: Finished task 1.0 in stage 71.0 (TID 345) in
3520 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:30 INFO TaskSetManager: Finished task 2.0 in stage 71.0 (TID 346) in
3290 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:30 INFO ShufflePartitionsUtil: For shuffle(70, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:31 INFO TaskSetManager: Starting task 1.0 in stage 74.0 (TID 355)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:31 INFO TaskSetManager: Finished task 3.0 in stage 71.0 (TID 347) in
3297 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:31 INFO TaskSetManager: Starting task 2.0 in stage 74.0 (TID 356)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:31 INFO TaskSetManager: Finished task 4.0 in stage 71.0 (TID 348) in
3287 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:31 INFO TaskSchedulerImpl: Removed TaskSet 71.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:31 INFO DAGScheduler: ShuffleMapStage 71 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 157.706 s
25/02/06 06:10:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:31 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 73, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108, ShuffleMapStage 87,
ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160, ShuffleMapStage 110)
25/02/06 06:10:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:31 INFO DAGScheduler: failed: Set()
25/02/06 06:10:31 INFO DAGScheduler: Registering RDD 1002 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 158
25/02/06 06:10:31 INFO DAGScheduler: Got map stage job 161 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:31 INFO DAGScheduler: Final stage: ShuffleMapStage 230
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 229)
25/02/06 06:10:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:31 INFO DAGScheduler: Submitting ShuffleMapStage 230
(MapPartitionsRDD[1002] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:31 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 230 (MapPartitionsRDD[1002] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:31 INFO TaskSchedulerImpl: Adding task set 230.0 with 9 tasks
resource profile 0
25/02/06 06:10:31 INFO TaskSetManager: TaskSet 230.0 using PreferredLocationsV1
25/02/06 06:10:31 INFO FairSchedulableBuilder: Added task set TaskSet_230.0 tasks
to pool 5400065285585087590
25/02/06 06:10:31 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098554) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098554)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:31 INFO TaskSetManager: Starting task 3.0 in stage 74.0 (TID 357)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:31 INFO TaskSetManager: Finished task 0.0 in stage 73.0 (TID 349) in
3211 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:31 INFO ShufflePartitionsUtil: For shuffle(51, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:32 INFO DAGScheduler: Registering RDD 1010 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 159
25/02/06 06:10:32 INFO DAGScheduler: Got map stage job 162 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:32 INFO DAGScheduler: Final stage: ShuffleMapStage 232
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:32 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
231, ShuffleMapStage 97)
25/02/06 06:10:32 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:32 INFO DAGScheduler: Submitting ShuffleMapStage 232
(MapPartitionsRDD[1010] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:32 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 232 (MapPartitionsRDD[1010] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:32 INFO TaskSchedulerImpl: Adding task set 232.0 with 9 tasks
resource profile 0
25/02/06 06:10:32 INFO TaskSetManager: TaskSet 232.0 using PreferredLocationsV1
25/02/06 06:10:32 INFO FairSchedulableBuilder: Added task set TaskSet_232.0 tasks
to pool 5400065285585087590
25/02/06 06:10:32 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097610) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097610)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:32 INFO TaskSetManager: Starting task 4.0 in stage 74.0 (TID 358)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:32 INFO TaskSetManager: Finished task 1.0 in stage 73.0 (TID 350) in
3823 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:33 INFO TaskSetManager: Starting task 0.0 in stage 75.0 (TID 359)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:33 INFO TaskSetManager: Finished task 2.0 in stage 73.0 (TID 351) in
3378 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:33 INFO TaskSetManager: Starting task 1.0 in stage 75.0 (TID 360)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:33 INFO TaskSetManager: Finished task 3.0 in stage 73.0 (TID 352) in
3440 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:34 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:34 INFO TaskSetManager: Starting task 2.0 in stage 75.0 (TID 361)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:34 INFO TaskSetManager: Finished task 0.0 in stage 74.0 (TID 354) in
3518 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:34 INFO TaskSetManager: Starting task 3.0 in stage 75.0 (TID 362)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:34 INFO TaskSetManager: Finished task 4.0 in stage 73.0 (TID 353) in
3574 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:34 INFO TaskSchedulerImpl: Removed TaskSet 73.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:34 INFO DAGScheduler: ShuffleMapStage 73 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 160.779 s
25/02/06 06:10:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:34 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 198,
ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 74, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:34 INFO DAGScheduler: failed: Set()
25/02/06 06:10:34 INFO TaskSetManager: Starting task 4.0 in stage 75.0 (TID 363)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:34 INFO TaskSetManager: Finished task 1.0 in stage 74.0 (TID 355) in
3673 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:34 INFO ShufflePartitionsUtil: For shuffle(73, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:34 INFO TaskSetManager: Starting task 0.0 in stage 76.0 (TID 364)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:34 INFO TaskSetManager: Finished task 2.0 in stage 74.0 (TID 356) in
3533 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:35 INFO TaskSetManager: Finished task 3.0 in stage 74.0 (TID 357) in
3484 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:35 INFO TaskSetManager: Starting task 1.0 in stage 76.0 (TID 365)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:35 INFO DAGScheduler: Registering RDD 1018 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 160
25/02/06 06:10:35 INFO DAGScheduler: Got map stage job 163 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:35 INFO DAGScheduler: Final stage: ShuffleMapStage 234
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:35 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
233, ShuffleMapStage 97)
25/02/06 06:10:35 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:35 INFO DAGScheduler: Submitting ShuffleMapStage 234
(MapPartitionsRDD[1018] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:35 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 234 (MapPartitionsRDD[1018] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:35 INFO TaskSchedulerImpl: Adding task set 234.0 with 9 tasks
resource profile 0
25/02/06 06:10:35 INFO TaskSetManager: TaskSet 234.0 using PreferredLocationsV1
25/02/06 06:10:35 INFO FairSchedulableBuilder: Added task set TaskSet_234.0 tasks
to pool 5400065285585087590
25/02/06 06:10:35 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099759) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099759)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:36 INFO TaskSetManager: Starting task 2.0 in stage 76.0 (TID 366)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:36 INFO TaskSetManager: Finished task 4.0 in stage 74.0 (TID 358) in
3661 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:36 INFO TaskSchedulerImpl: Removed TaskSet 74.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:36 INFO DAGScheduler: ShuffleMapStage 74 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 162.928 s
25/02/06 06:10:36 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:36 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 75,
ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98,
ShuffleMapStage 77, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 198,
ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:36 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:36 INFO DAGScheduler: failed: Set()
25/02/06 06:10:36 INFO ShufflePartitionsUtil: For shuffle(74, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:36 INFO TaskSetManager: Starting task 3.0 in stage 76.0 (TID 367)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:36 INFO TaskSetManager: Finished task 0.0 in stage 75.0 (TID 359) in
3380 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:37 INFO TaskSetManager: Starting task 4.0 in stage 76.0 (TID 368)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:37 INFO TaskSetManager: Finished task 1.0 in stage 75.0 (TID 360) in
3354 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:37 INFO DAGScheduler: Registering RDD 1026 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 161
25/02/06 06:10:37 INFO DAGScheduler: Got map stage job 164 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:37 INFO DAGScheduler: Final stage: ShuffleMapStage 236
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
235, ShuffleMapStage 97)
25/02/06 06:10:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:37 INFO DAGScheduler: Submitting ShuffleMapStage 236
(MapPartitionsRDD[1026] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:37 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 236 (MapPartitionsRDD[1026] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:37 INFO TaskSchedulerImpl: Adding task set 236.0 with 9 tasks
resource profile 0
25/02/06 06:10:37 INFO TaskSetManager: TaskSet 236.0 using PreferredLocationsV1
25/02/06 06:10:37 INFO FairSchedulableBuilder: Added task set TaskSet_236.0 tasks
to pool 5400065285585087590
25/02/06 06:10:37 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097744) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097744)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:37 INFO TaskSetManager: Starting task 0.0 in stage 81.0 (TID 369)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:37 INFO TaskSetManager: Finished task 2.0 in stage 75.0 (TID 361) in
3368 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:37 INFO TaskSetManager: Starting task 1.0 in stage 81.0 (TID 370)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:37 INFO TaskSetManager: Finished task 3.0 in stage 75.0 (TID 362) in
3493 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:38 INFO TaskSetManager: Starting task 2.0 in stage 81.0 (TID 371)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:38 INFO TaskSetManager: Finished task 0.0 in stage 76.0 (TID 364) in
3495 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:38 INFO TaskSetManager: Starting task 3.0 in stage 81.0 (TID 372)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:38 INFO TaskSetManager: Finished task 4.0 in stage 75.0 (TID 363) in
3684 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:38 INFO TaskSchedulerImpl: Removed TaskSet 75.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:38 INFO DAGScheduler: ShuffleMapStage 75 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 164.775 s
25/02/06 06:10:38 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:38 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 76, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:38 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:38 INFO DAGScheduler: failed: Set()
25/02/06 06:10:38 INFO TaskSetManager: Starting task 4.0 in stage 81.0 (TID 373)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:38 INFO TaskSetManager: Finished task 1.0 in stage 76.0 (TID 365) in
3240 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:38 INFO ShufflePartitionsUtil: For shuffle(76, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:39 INFO DAGScheduler: Registering RDD 1034 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 162
25/02/06 06:10:39 INFO DAGScheduler: Got map stage job 165 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:39 INFO DAGScheduler: Final stage: ShuffleMapStage 238
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:39 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
237, ShuffleMapStage 97)
25/02/06 06:10:39 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:39 INFO DAGScheduler: Submitting ShuffleMapStage 238
(MapPartitionsRDD[1034] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:39 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 238 (MapPartitionsRDD[1034] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:39 INFO TaskSchedulerImpl: Adding task set 238.0 with 9 tasks
resource profile 0
25/02/06 06:10:39 INFO TaskSetManager: TaskSet 238.0 using PreferredLocationsV1
25/02/06 06:10:39 INFO FairSchedulableBuilder: Added task set TaskSet_238.0 tasks
to pool 5400065285585087590
25/02/06 06:10:39 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099887) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099887)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:40 INFO TaskSetManager: Starting task 0.0 in stage 77.0 (TID 374)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:40 INFO TaskSetManager: Finished task 2.0 in stage 76.0 (TID 366) in
3627 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:40 INFO TaskSetManager: Starting task 1.0 in stage 77.0 (TID 375)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:40 INFO TaskSetManager: Finished task 4.0 in stage 76.0 (TID 368) in
3114 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:40 INFO TaskSetManager: Starting task 2.0 in stage 77.0 (TID 376)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:40 INFO TaskSetManager: Finished task 3.0 in stage 76.0 (TID 367) in
3339 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:40 INFO TaskSchedulerImpl: Removed TaskSet 76.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:40 INFO DAGScheduler: ShuffleMapStage 76 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 166.629 s
25/02/06 06:10:40 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:40 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 81, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:40 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:40 INFO DAGScheduler: failed: Set()
25/02/06 06:10:40 INFO ShufflePartitionsUtil: For shuffle(77, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:41 INFO TaskSetManager: Starting task 3.0 in stage 77.0 (TID 377)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:41 INFO TaskSetManager: Finished task 0.0 in stage 81.0 (TID 369) in
3425 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:41 INFO TaskSetManager: Starting task 4.0 in stage 77.0 (TID 378)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:41 INFO TaskSetManager: Finished task 1.0 in stage 81.0 (TID 370) in
3256 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:41 INFO DAGScheduler: Registering RDD 1042 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 163
25/02/06 06:10:41 INFO DAGScheduler: Got map stage job 166 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:41 INFO DAGScheduler: Final stage: ShuffleMapStage 240
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
239, ShuffleMapStage 97)
25/02/06 06:10:41 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:41 INFO DAGScheduler: Submitting ShuffleMapStage 240
(MapPartitionsRDD[1042] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:41 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 240 (MapPartitionsRDD[1042] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:41 INFO TaskSchedulerImpl: Adding task set 240.0 with 9 tasks
resource profile 0
25/02/06 06:10:41 INFO TaskSetManager: TaskSet 240.0 using PreferredLocationsV1
25/02/06 06:10:41 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098652) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098652)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:41 INFO FairSchedulableBuilder: Added task set TaskSet_240.0 tasks
to pool 5400065285585087590
25/02/06 06:10:41 INFO TaskSetManager: Starting task 0.0 in stage 78.0 (TID 379)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:41 INFO TaskSetManager: Finished task 2.0 in stage 81.0 (TID 371) in
3310 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:41 INFO TaskSetManager: Starting task 1.0 in stage 78.0 (TID 380)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:41 INFO TaskSetManager: Finished task 3.0 in stage 81.0 (TID 372) in
3226 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:41 INFO TaskSetManager: Starting task 2.0 in stage 78.0 (TID 381)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:41 INFO TaskSetManager: Finished task 4.0 in stage 81.0 (TID 373) in
3140 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:41 INFO TaskSchedulerImpl: Removed TaskSet 81.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:41 INFO DAGScheduler: ShuffleMapStage 81 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 167.946 s
25/02/06 06:10:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:41 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 77, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:41 INFO DAGScheduler: failed: Set()
25/02/06 06:10:41 INFO ShufflePartitionsUtil: For shuffle(78, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:42 INFO DAGScheduler: Registering RDD 1050 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 164
25/02/06 06:10:42 INFO DAGScheduler: Got map stage job 167 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:42 INFO DAGScheduler: Final stage: ShuffleMapStage 242
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:42 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
241, ShuffleMapStage 97)
25/02/06 06:10:42 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:42 INFO DAGScheduler: Submitting ShuffleMapStage 242
(MapPartitionsRDD[1050] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:42 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 242 (MapPartitionsRDD[1050] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:42 INFO TaskSchedulerImpl: Adding task set 242.0 with 9 tasks
resource profile 0
25/02/06 06:10:42 INFO TaskSetManager: TaskSet 242.0 using PreferredLocationsV1
25/02/06 06:10:42 INFO FairSchedulableBuilder: Added task set TaskSet_242.0 tasks
to pool 5400065285585087590
25/02/06 06:10:42 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099666) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099666)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:42 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-
20250206055605-0000/5 on worker-20250206061040-10.183.233.191-35573
(10.183.233.191:35573) with 4 core(s)
25/02/06 06:10:42 INFO StandaloneSchedulerBackend: Granted executor ID app-
20250206055605-0000/5 on hostPort 10.183.233.191:35573 with 4 core(s), 5.3 GiB RAM
25/02/06 06:10:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:43 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-
20250206055605-0000/6 on worker-20250206061041-10.183.246.72-46855
(10.183.246.72:46855) with 4 core(s)
25/02/06 06:10:43 INFO StandaloneSchedulerBackend: Granted executor ID app-
20250206055605-0000/6 on hostPort 10.183.246.72:46855 with 4 core(s), 5.3 GiB RAM
25/02/06 06:10:43 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-
20250206055605-0000/5 is now RUNNING
25/02/06 06:10:43 INFO TaskSetManager: Starting task 3.0 in stage 78.0 (TID 382)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:43 INFO TaskSetManager: Finished task 2.0 in stage 77.0 (TID 376) in
3222 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:43 INFO TaskSetManager: Starting task 4.0 in stage 78.0 (TID 383)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:43 INFO TaskSetManager: Finished task 1.0 in stage 77.0 (TID 375) in
3312 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:43 INFO TaskSetManager: Starting task 0.0 in stage 79.0 (TID 384)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:43 INFO TaskSetManager: Finished task 0.0 in stage 77.0 (TID 374) in
3422 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:43 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-
20250206055605-0000/6 is now RUNNING
25/02/06 06:10:44 INFO TaskSetManager: Starting task 1.0 in stage 79.0 (TID 385)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:44 INFO TaskSetManager: Finished task 4.0 in stage 77.0 (TID 378) in
3164 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:44 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-
20250206055605-0000/7 on worker-20250206061042-10.183.249.197-34271
(10.183.249.197:34271) with 4 core(s)
25/02/06 06:10:44 INFO StandaloneSchedulerBackend: Granted executor ID app-
20250206055605-0000/7 on hostPort 10.183.249.197:34271 with 4 core(s), 5.3 GiB RAM
25/02/06 06:10:44 INFO TaskSetManager: Starting task 2.0 in stage 79.0 (TID 386)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:44 INFO TaskSetManager: Finished task 3.0 in stage 77.0 (TID 377) in
3340 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:44 INFO TaskSchedulerImpl: Removed TaskSet 77.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:44 INFO DAGScheduler: ShuffleMapStage 77 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 170.809 s
25/02/06 06:10:44 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:44 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 80, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 82, ShuffleMapStage 184, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 78, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:44 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:44 INFO DAGScheduler: failed: Set()
25/02/06 06:10:44 INFO ShufflePartitionsUtil: For shuffle(79, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:44 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-
20250206055605-0000/7 is now RUNNING
25/02/06 06:10:44 INFO TaskSetManager: Finished task 2.0 in stage 78.0 (TID 381) in
3311 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:44 INFO TaskSetManager: Starting task 3.0 in stage 79.0 (TID 387)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:45 INFO TaskSetManager: Starting task 4.0 in stage 79.0 (TID 388)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:45 INFO TaskSetManager: Finished task 0.0 in stage 78.0 (TID 379) in
3616 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:45 INFO TaskSetManager: Starting task 0.0 in stage 80.0 (TID 389)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:45 INFO TaskSetManager: Finished task 1.0 in stage 78.0 (TID 380) in
3624 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:45 INFO DAGScheduler: Registering RDD 1058 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 165
25/02/06 06:10:45 INFO DAGScheduler: Got map stage job 168 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:45 INFO DAGScheduler: Final stage: ShuffleMapStage 244
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:45 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
243, ShuffleMapStage 97)
25/02/06 06:10:45 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:45 INFO DAGScheduler: Submitting ShuffleMapStage 244
(MapPartitionsRDD[1058] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:45 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 244 (MapPartitionsRDD[1058] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:45 INFO TaskSchedulerImpl: Adding task set 244.0 with 9 tasks
resource profile 0
25/02/06 06:10:45 INFO TaskSetManager: TaskSet 244.0 using PreferredLocationsV1
25/02/06 06:10:45 INFO FairSchedulableBuilder: Added task set TaskSet_244.0 tasks
to pool 5400065285585087590
25/02/06 06:10:45 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099757) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099757)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:46 INFO TaskSetManager: Starting task 1.0 in stage 80.0 (TID 390)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:46 INFO TaskSetManager: Finished task 3.0 in stage 78.0 (TID 382) in
3421 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:47 INFO TaskSetManager: Starting task 2.0 in stage 80.0 (TID 391)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:47 INFO TaskSetManager: Finished task 0.0 in stage 79.0 (TID 384) in
3492 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:47 INFO TaskSetManager: Starting task 3.0 in stage 80.0 (TID 392)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:47 INFO TaskSetManager: Finished task 4.0 in stage 78.0 (TID 383) in
3672 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:47 INFO TaskSchedulerImpl: Removed TaskSet 78.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:47 INFO DAGScheduler: ShuffleMapStage 78 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 173.535 s
25/02/06 06:10:47 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:47 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 80, ShuffleMapStage 182,
ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 82, ShuffleMapStage 184,
ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 85,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 108,
ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 79, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:47 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:47 INFO DAGScheduler: failed: Set()
25/02/06 06:10:47 INFO ShufflePartitionsUtil: For shuffle(48, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:47 INFO TaskSetManager: Starting task 4.0 in stage 80.0 (TID 393)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:47 INFO TaskSetManager: Finished task 1.0 in stage 79.0 (TID 385) in
3622 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:48 INFO DAGScheduler: Registering RDD 1066 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 166
25/02/06 06:10:48 INFO DAGScheduler: Got map stage job 169 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:48 INFO DAGScheduler: Final stage: ShuffleMapStage 246
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:48 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
245, ShuffleMapStage 97)
25/02/06 06:10:48 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:48 INFO DAGScheduler: Submitting ShuffleMapStage 246
(MapPartitionsRDD[1066] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:48 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 246 (MapPartitionsRDD[1066] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:48 INFO TaskSchedulerImpl: Adding task set 246.0 with 9 tasks
resource profile 0
25/02/06 06:10:48 INFO TaskSetManager: TaskSet 246.0 using PreferredLocationsV1
25/02/06 06:10:48 INFO FairSchedulableBuilder: Added task set TaskSet_246.0 tasks
to pool 5400065285585087590
25/02/06 06:10:48 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100890) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100890)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:48 INFO TaskSetManager: Starting task 0.0 in stage 82.0 (TID 394)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:48 INFO TaskSetManager: Finished task 2.0 in stage 79.0 (TID 386) in
3664 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:48 INFO TaskSetManager: Starting task 1.0 in stage 82.0 (TID 395)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:48 INFO TaskSetManager: Finished task 3.0 in stage 79.0 (TID 387) in
3295 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:48 INFO TaskSetManager: Finished task 0.0 in stage 80.0 (TID 389) in
3174 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:48 INFO TaskSetManager: Starting task 2.0 in stage 82.0 (TID 396)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:49 INFO TaskSetManager: Starting task 3.0 in stage 82.0 (TID 397)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:49 INFO TaskSetManager: Finished task 4.0 in stage 79.0 (TID 388) in
3795 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:49 INFO TaskSchedulerImpl: Removed TaskSet 79.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:49 INFO DAGScheduler: ShuffleMapStage 79 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 175.363 s
25/02/06 06:10:49 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:49 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 84, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 80,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 85, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:49 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:49 INFO DAGScheduler: failed: Set()
25/02/06 06:10:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:49 INFO ShufflePartitionsUtil: For shuffle(52, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:49 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099710) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099710)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:49 INFO DAGScheduler: Registering RDD 1074 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 167
25/02/06 06:10:49 INFO DAGScheduler: Got map stage job 170 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:49 INFO DAGScheduler: Final stage: ShuffleMapStage 248
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:49 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 247)
25/02/06 06:10:49 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:49 INFO DAGScheduler: Submitting ShuffleMapStage 248
(MapPartitionsRDD[1074] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:49 INFO TaskSetManager: Finished task 1.0 in stage 80.0 (TID 390) in
3023 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:49 INFO TaskSetManager: Starting task 4.0 in stage 82.0 (TID 398)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:49 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 248 (MapPartitionsRDD[1074] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:49 INFO TaskSchedulerImpl: Adding task set 248.0 with 9 tasks
resource profile 0
25/02/06 06:10:49 INFO TaskSetManager: TaskSet 248.0 using PreferredLocationsV1
25/02/06 06:10:49 INFO FairSchedulableBuilder: Added task set TaskSet_248.0 tasks
to pool 5400065285585087590
25/02/06 06:10:50 INFO TaskSetManager: Finished task 3.0 in stage 80.0 (TID 392) in
3211 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:50 INFO TaskSetManager: Starting task 0.0 in stage 83.0 (TID 399)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:50 INFO TaskSetManager: Starting task 1.0 in stage 83.0 (TID 400)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:50 INFO TaskSetManager: Finished task 2.0 in stage 80.0 (TID 391) in
3382 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:51 INFO TaskSetManager: Starting task 2.0 in stage 83.0 (TID 401)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:51 INFO TaskSetManager: Finished task 4.0 in stage 80.0 (TID 393) in
3184 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:51 INFO TaskSchedulerImpl: Removed TaskSet 80.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:51 INFO DAGScheduler: ShuffleMapStage 80 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 177.419 s
25/02/06 06:10:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:51 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 88,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 84, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 82,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 85, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:51 INFO DAGScheduler: failed: Set()
25/02/06 06:10:51 INFO TaskSetManager: Starting task 3.0 in stage 83.0 (TID 402)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:51 INFO TaskSetManager: Finished task 1.0 in stage 82.0 (TID 395) in
3208 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:51 INFO ShufflePartitionsUtil: For shuffle(54, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:51 INFO TaskSetManager: Starting task 4.0 in stage 83.0 (TID 403)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:51 INFO TaskSetManager: Finished task 0.0 in stage 82.0 (TID 394) in
3447 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:51 INFO TaskSetManager: Starting task 0.0 in stage 84.0 (TID 404)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:51 INFO TaskSetManager: Finished task 2.0 in stage 82.0 (TID 396) in
3194 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:51 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099901) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099901)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:52 INFO DAGScheduler: Registering RDD 1082 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 168
25/02/06 06:10:52 INFO DAGScheduler: Got map stage job 171 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:52 INFO DAGScheduler: Final stage: ShuffleMapStage 250
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:52 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
249, ShuffleMapStage 97)
25/02/06 06:10:52 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:52 INFO DAGScheduler: Submitting ShuffleMapStage 250
(MapPartitionsRDD[1082] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:52 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 250 (MapPartitionsRDD[1082] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:52 INFO TaskSchedulerImpl: Adding task set 250.0 with 9 tasks
resource profile 0
25/02/06 06:10:52 INFO TaskSetManager: TaskSet 250.0 using PreferredLocationsV1
25/02/06 06:10:52 INFO FairSchedulableBuilder: Added task set TaskSet_250.0 tasks
to pool 5400065285585087590
25/02/06 06:10:52 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:52 INFO TaskSetManager: Starting task 1.0 in stage 84.0 (TID 405)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:52 INFO TaskSetManager: Finished task 3.0 in stage 82.0 (TID 397) in
3472 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:53 INFO TaskSetManager: Starting task 2.0 in stage 84.0 (TID 406)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:10:53 INFO TaskSetManager: Finished task 4.0 in stage 82.0 (TID 398) in
3147 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:10:53 INFO TaskSchedulerImpl: Removed TaskSet 82.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:53 INFO DAGScheduler: ShuffleMapStage 82 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 179.374 s
25/02/06 06:10:53 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:53 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 83, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 250, ShuffleMapStage 98, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 85, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:53 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:53 INFO DAGScheduler: failed: Set()
25/02/06 06:10:53 INFO ShufflePartitionsUtil: For shuffle(82, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:53 INFO TaskSetManager: Starting task 3.0 in stage 84.0 (TID 407)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:53 INFO TaskSetManager: Finished task 0.0 in stage 83.0 (TID 399) in
3274 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:53 INFO TaskSetManager: Finished task 1.0 in stage 83.0 (TID 400) in
3510 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:53 INFO TaskSetManager: Starting task 4.0 in stage 84.0 (TID 408)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:53 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097895) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097895)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:53 INFO DAGScheduler: Registering RDD 1090 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 169
25/02/06 06:10:53 INFO DAGScheduler: Got map stage job 172 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:53 INFO DAGScheduler: Final stage: ShuffleMapStage 252
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:53 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
251, ShuffleMapStage 97)
25/02/06 06:10:53 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:53 INFO DAGScheduler: Submitting ShuffleMapStage 252
(MapPartitionsRDD[1090] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:53 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 252 (MapPartitionsRDD[1090] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:53 INFO TaskSchedulerImpl: Adding task set 252.0 with 9 tasks
resource profile 0
25/02/06 06:10:53 INFO TaskSetManager: TaskSet 252.0 using PreferredLocationsV1
25/02/06 06:10:53 INFO FairSchedulableBuilder: Added task set TaskSet_252.0 tasks
to pool 5400065285585087590
25/02/06 06:10:54 INFO TaskSetManager: Starting task 0.0 in stage 85.0 (TID 409)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:54 INFO TaskSetManager: Finished task 2.0 in stage 83.0 (TID 401) in
3284 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:54 INFO TaskSetManager: Starting task 1.0 in stage 85.0 (TID 410)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:54 INFO TaskSetManager: Finished task 3.0 in stage 83.0 (TID 402) in
3151 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:54 INFO TaskSetManager: Starting task 2.0 in stage 85.0 (TID 411)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:54 INFO TaskSetManager: Finished task 0.0 in stage 84.0 (TID 404) in
3131 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:10:54 INFO TaskSetManager: Starting task 3.0 in stage 85.0 (TID 412)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:10:54 INFO TaskSetManager: Finished task 4.0 in stage 83.0 (TID 403) in
3242 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:54 INFO TaskSchedulerImpl: Removed TaskSet 83.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:54 INFO DAGScheduler: ShuffleMapStage 83 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 181.102 s
25/02/06 06:10:54 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:54 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 104,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148, ShuffleMapStage 106,
ShuffleMapStage 250, ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 252, ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 84,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 85, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:54 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:54 INFO DAGScheduler: failed: Set()
25/02/06 06:10:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:55 INFO ShufflePartitionsUtil: For shuffle(75, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:55 INFO TaskSetManager: Starting task 4.0 in stage 85.0 (TID 413)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:10:55 INFO TaskSetManager: Finished task 1.0 in stage 84.0 (TID 405) in
3341 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:10:55 INFO DAGScheduler: Registering RDD 1098 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 170
25/02/06 06:10:55 INFO DAGScheduler: Got map stage job 173 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:55 INFO DAGScheduler: Final stage: ShuffleMapStage 254
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:55 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
253, ShuffleMapStage 97)
25/02/06 06:10:55 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:55 INFO DAGScheduler: Submitting ShuffleMapStage 254
(MapPartitionsRDD[1098] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:55 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099946) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099946)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:55 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 254 (MapPartitionsRDD[1098] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:55 INFO TaskSchedulerImpl: Adding task set 254.0 with 9 tasks
resource profile 0
25/02/06 06:10:55 INFO TaskSetManager: TaskSet 254.0 using PreferredLocationsV1
25/02/06 06:10:55 INFO FairSchedulableBuilder: Added task set TaskSet_254.0 tasks
to pool 5400065285585087590
25/02/06 06:10:56 INFO TaskSetManager: Starting task 0.0 in stage 86.0 (TID 414)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:10:56 INFO TaskSetManager: Finished task 2.0 in stage 84.0 (TID 406) in
3140 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:10:56 INFO TaskSetManager: Starting task 1.0 in stage 86.0 (TID 415)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:10:56 INFO TaskSetManager: Finished task 3.0 in stage 84.0 (TID 407) in
3152 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:10:57 INFO TaskSetManager: Starting task 2.0 in stage 86.0 (TID 416)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:57 INFO TaskSetManager: Finished task 4.0 in stage 84.0 (TID 408) in
3409 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:57 INFO TaskSchedulerImpl: Removed TaskSet 84.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:57 INFO DAGScheduler: ShuffleMapStage 84 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 183.657 s
25/02/06 06:10:57 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:57 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 250, ShuffleMapStage 98, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 100, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248,
ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 86,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 85, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:57 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:57 INFO DAGScheduler: failed: Set()
25/02/06 06:10:57 INFO TaskSetManager: Finished task 0.0 in stage 85.0 (TID 409) in
3216 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:10:57 INFO TaskSetManager: Starting task 3.0 in stage 86.0 (TID 417)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:57 INFO ShufflePartitionsUtil: For shuffle(80, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:10:58 INFO TaskSetManager: Starting task 4.0 in stage 86.0 (TID 418)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:10:58 INFO TaskSetManager: Finished task 1.0 in stage 85.0 (TID 410) in
3447 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:10:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:10:58 INFO DAGScheduler: Registering RDD 1106 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 171
25/02/06 06:10:58 INFO DAGScheduler: Got map stage job 174 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:10:58 INFO DAGScheduler: Final stage: ShuffleMapStage 256
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:10:58 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
255, ShuffleMapStage 97)
25/02/06 06:10:58 INFO DAGScheduler: Missing parents: List()
25/02/06 06:10:58 INFO DAGScheduler: Submitting ShuffleMapStage 256
(MapPartitionsRDD[1106] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:10:58 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 256 (MapPartitionsRDD[1106] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:10:58 INFO TaskSchedulerImpl: Adding task set 256.0 with 9 tasks
resource profile 0
25/02/06 06:10:58 INFO TaskSetManager: TaskSet 256.0 using PreferredLocationsV1
25/02/06 06:10:58 INFO FairSchedulableBuilder: Added task set TaskSet_256.0 tasks
to pool 5400065285585087590
25/02/06 06:10:58 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098826) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098826)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:10:58 INFO TaskSetManager: Starting task 0.0 in stage 87.0 (TID 419)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:10:58 INFO TaskSetManager: Finished task 2.0 in stage 85.0 (TID 411) in
3726 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:10:58 INFO TaskSetManager: Starting task 1.0 in stage 87.0 (TID 420)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:10:58 INFO TaskSetManager: Finished task 3.0 in stage 85.0 (TID 412) in
3899 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:10:59 INFO TaskSetManager: Starting task 2.0 in stage 87.0 (TID 421)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:10:59 INFO TaskSetManager: Finished task 4.0 in stage 85.0 (TID 413) in
3860 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:10:59 INFO TaskSchedulerImpl: Removed TaskSet 85.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:10:59 INFO DAGScheduler: ShuffleMapStage 85 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 185.976 s
25/02/06 06:10:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:10:59 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250, ShuffleMapStage 98,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 88, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 86, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:10:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:10:59 INFO DAGScheduler: failed: Set()
25/02/06 06:10:59 INFO TaskSetManager: Starting task 3.0 in stage 87.0 (TID 422)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:10:59 INFO TaskSetManager: Finished task 0.0 in stage 86.0 (TID 414) in
3776 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:00 INFO ShufflePartitionsUtil: For shuffle(81, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:00 INFO TaskSetManager: Starting task 4.0 in stage 87.0 (TID 423)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:00 INFO TaskSetManager: Finished task 1.0 in stage 86.0 (TID 415) in
3313 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:11:00 INFO DataSourceFactory$: DataSource Jdbc URL:
jdbc:mariadb://md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com:3306/
organization8517174019523704?
useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertif
icate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt
25/02/06 06:11:00 INFO HikariDataSource: metastore-monitor - Starting...
25/02/06 06:11:00 INFO HikariDataSource: metastore-monitor - Start completed.
25/02/06 06:11:00 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099858) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099858)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:00 INFO DAGScheduler: Registering RDD 1114 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 172
25/02/06 06:11:00 INFO DAGScheduler: Got map stage job 175 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:00 INFO DAGScheduler: Final stage: ShuffleMapStage 258
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
257, ShuffleMapStage 97)
25/02/06 06:11:00 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:00 INFO DAGScheduler: Submitting ShuffleMapStage 258
(MapPartitionsRDD[1114] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:00 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 258 (MapPartitionsRDD[1114] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:00 INFO TaskSchedulerImpl: Adding task set 258.0 with 9 tasks
resource profile 0
25/02/06 06:11:00 INFO TaskSetManager: TaskSet 258.0 using PreferredLocationsV1
25/02/06 06:11:00 INFO FairSchedulableBuilder: Added task set TaskSet_258.0 tasks
to pool 5400065285585087590
25/02/06 06:11:01 INFO TaskSetManager: Starting task 0.0 in stage 88.0 (TID 424)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:01 INFO TaskSetManager: Finished task 3.0 in stage 86.0 (TID 417) in
3431 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:11:01 INFO HikariDataSource: metastore-monitor - Shutdown initiated...
25/02/06 06:11:01 INFO HikariDataSource: metastore-monitor - Shutdown completed.
25/02/06 06:11:01 INFO MetastoreMonitor: Metastore healthcheck successful
(connection duration = 371 milliseconds)
25/02/06 06:11:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:01 INFO TaskSetManager: Starting task 1.0 in stage 88.0 (TID 425)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:01 INFO TaskSetManager: Finished task 2.0 in stage 86.0 (TID 416) in
3763 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:11:01 INFO TaskSetManager: Starting task 2.0 in stage 88.0 (TID 426)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:01 INFO TaskSetManager: Finished task 4.0 in stage 86.0 (TID 418) in
3793 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:11:01 INFO TaskSchedulerImpl: Removed TaskSet 86.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:01 INFO DAGScheduler: ShuffleMapStage 86 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 188.100 s
25/02/06 06:11:01 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:01 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 88,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 108, ShuffleMapStage 87, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:11:01 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:01 INFO DAGScheduler: failed: Set()
25/02/06 06:11:02 INFO TaskSetManager: Starting task 3.0 in stage 88.0 (TID 427)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:02 INFO TaskSetManager: Finished task 0.0 in stage 87.0 (TID 419) in
3545 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:11:02 INFO TaskSetManager: Starting task 4.0 in stage 88.0 (TID 428)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:02 INFO TaskSetManager: Finished task 1.0 in stage 87.0 (TID 420) in
3357 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:02 INFO ShufflePartitionsUtil: For shuffle(83, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:02 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097768) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097768)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:02 INFO DAGScheduler: Registering RDD 1122 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 173
25/02/06 06:11:02 INFO DAGScheduler: Got map stage job 176 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:02 INFO DAGScheduler: Final stage: ShuffleMapStage 260
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:02 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
259, ShuffleMapStage 97)
25/02/06 06:11:02 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:02 INFO DAGScheduler: Submitting ShuffleMapStage 260
(MapPartitionsRDD[1122] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:02 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 260 (MapPartitionsRDD[1122] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:02 INFO TaskSchedulerImpl: Adding task set 260.0 with 9 tasks
resource profile 0
25/02/06 06:11:02 INFO TaskSetManager: TaskSet 260.0 using PreferredLocationsV1
25/02/06 06:11:02 INFO FairSchedulableBuilder: Added task set TaskSet_260.0 tasks
to pool 5400065285585087590
25/02/06 06:11:03 INFO TaskSetManager: Starting task 0.0 in stage 90.0 (TID 429)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:03 INFO TaskSetManager: Finished task 2.0 in stage 87.0 (TID 421) in
3646 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:11:03 INFO TaskSetManager: Starting task 1.0 in stage 90.0 (TID 430)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:03 INFO TaskSetManager: Finished task 3.0 in stage 87.0 (TID 422) in
3431 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:11:03 INFO TaskSetManager: Starting task 2.0 in stage 90.0 (TID 431)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:03 INFO TaskSetManager: Finished task 4.0 in stage 87.0 (TID 423) in
3413 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:11:03 INFO TaskSchedulerImpl: Removed TaskSet 87.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:03 INFO DAGScheduler: ShuffleMapStage 87 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 189.804 s
25/02/06 06:11:03 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:03 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 88,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 110)
25/02/06 06:11:03 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:03 INFO DAGScheduler: failed: Set()
25/02/06 06:11:03 INFO ShufflePartitionsUtil: For shuffle(84, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:04 INFO DAGScheduler: Registering RDD 1130 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 174
25/02/06 06:11:04 INFO DAGScheduler: Got map stage job 177 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:04 INFO DAGScheduler: Final stage: ShuffleMapStage 262
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
261, ShuffleMapStage 97)
25/02/06 06:11:04 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:04 INFO DAGScheduler: Submitting ShuffleMapStage 262
(MapPartitionsRDD[1130] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:04 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099825) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099825)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:04 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 262 (MapPartitionsRDD[1130] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:04 INFO TaskSchedulerImpl: Adding task set 262.0 with 9 tasks
resource profile 0
25/02/06 06:11:04 INFO TaskSetManager: TaskSet 262.0 using PreferredLocationsV1
25/02/06 06:11:04 INFO FairSchedulableBuilder: Added task set TaskSet_262.0 tasks
to pool 5400065285585087590
25/02/06 06:11:04 INFO TaskSetManager: Starting task 3.0 in stage 90.0 (TID 432)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:04 INFO TaskSetManager: Finished task 0.0 in stage 88.0 (TID 424) in
3495 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:04 INFO TaskSetManager: Starting task 4.0 in stage 90.0 (TID 433)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:04 INFO TaskSetManager: Finished task 1.0 in stage 88.0 (TID 425) in
3481 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:05 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint:
Registered ()executor NettyRpcEndpointRef(spark-client://Executor)
(10.183.233.191:55538) with ID 5, ResourceProfileId 0
25/02/06 06:11:05 INFO TaskSetManager: Starting task 0.0 in stage 89.0 (TID 434)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Finished task 2.0 in stage 88.0 (TID 426) in
3393 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:11:05 INFO BlockManagerMasterEndpoint: Registering block manager
10.183.233.191:42923 with 18.4 GiB RAM, BlockManagerId(5, 10.183.233.191, 42923,
None)
25/02/06 06:11:05 INFO TaskSetManager: Starting task 1.0 in stage 89.0 (TID 435)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Finished task 3.0 in stage 88.0 (TID 427) in
3393 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:11:05 INFO TaskSetManager: Starting task 2.0 in stage 89.0 (TID 436)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Starting task 3.0 in stage 89.0 (TID 437)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Starting task 4.0 in stage 89.0 (TID 438)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Starting task 0.0 in stage 91.0 (TID 439)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Starting task 1.0 in stage 91.0 (TID 440)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:05 INFO TaskSetManager: Finished task 4.0 in stage 88.0 (TID 428) in
3763 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:11:05 INFO TaskSchedulerImpl: Removed TaskSet 88.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:05 INFO DAGScheduler: ShuffleMapStage 88 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 192.066 s
25/02/06 06:11:05 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:05 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 90, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226,
ShuffleMapStage 116, ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 110)
25/02/06 06:11:05 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:05 INFO DAGScheduler: failed: Set()
25/02/06 06:11:06 INFO ShufflePartitionsUtil: For shuffle(86, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:06 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint:
Registered ()executor NettyRpcEndpointRef(spark-client://Executor)
(10.183.246.72:39896) with ID 6, ResourceProfileId 0
25/02/06 06:11:06 INFO BlockManagerMasterEndpoint: Registering block manager
10.183.246.72:43231 with 18.4 GiB RAM, BlockManagerId(6, 10.183.246.72, 43231,
None)
25/02/06 06:11:06 INFO TaskSetManager: Starting task 2.0 in stage 91.0 (TID 441)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:06 INFO TaskSetManager: Finished task 2.0 in stage 90.0 (TID 431) in
3353 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:06 INFO TaskSetManager: Starting task 3.0 in stage 91.0 (TID 442)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:06 INFO TaskSetManager: Starting task 4.0 in stage 91.0 (TID 443)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:06 INFO TaskSetManager: Starting task 0.0 in stage 92.0 (TID 444)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:06 INFO TaskSetManager: Starting task 1.0 in stage 92.0 (TID 445)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:06 INFO DAGScheduler: Registering RDD 1138 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 175
25/02/06 06:11:06 INFO DAGScheduler: Got map stage job 178 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:06 INFO DAGScheduler: Final stage: ShuffleMapStage 264
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:06 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
263, ShuffleMapStage 97)
25/02/06 06:11:06 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:06 INFO DAGScheduler: Submitting ShuffleMapStage 264
(MapPartitionsRDD[1138] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:06 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 264 (MapPartitionsRDD[1138] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:06 INFO TaskSchedulerImpl: Adding task set 264.0 with 9 tasks
resource profile 0
25/02/06 06:11:06 INFO TaskSetManager: TaskSet 264.0 using PreferredLocationsV1
25/02/06 06:11:06 INFO FairSchedulableBuilder: Added task set TaskSet_264.0 tasks
to pool 5400065285585087590
25/02/06 06:11:06 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097836) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097836)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:06 INFO TaskSetManager: Starting task 2.0 in stage 92.0 (TID 446)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:06 INFO TaskSetManager: Finished task 1.0 in stage 90.0 (TID 430) in
3577 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:11:07 INFO TaskSetManager: Starting task 3.0 in stage 92.0 (TID 447)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:07 INFO TaskSetManager: Finished task 0.0 in stage 90.0 (TID 429) in
3673 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:11:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:07 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint:
Registered ()executor NettyRpcEndpointRef(spark-client://Executor)
(10.183.249.197:36834) with ID 7, ResourceProfileId 0
25/02/06 06:11:07 INFO BlockManagerMasterEndpoint: Registering block manager
10.183.249.197:40023 with 18.4 GiB RAM, BlockManagerId(7, 10.183.249.197, 40023,
None)
25/02/06 06:11:07 INFO TaskSetManager: Starting task 4.0 in stage 92.0 (TID 448)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:07 INFO TaskSetManager: Starting task 0.0 in stage 93.0 (TID 449)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:07 INFO TaskSetManager: Starting task 1.0 in stage 93.0 (TID 450)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:07 INFO TaskSetManager: Starting task 2.0 in stage 93.0 (TID 451)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:07 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.233.191:42923 (size: 18.4 KiB, free: 18.4 GiB)
25/02/06 06:11:07 INFO TaskSetManager: Starting task 3.0 in stage 93.0 (TID 452)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:07 INFO TaskSetManager: Finished task 4.0 in stage 90.0 (TID 433) in
3306 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:11:08 INFO TaskSetManager: Starting task 4.0 in stage 93.0 (TID 453)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:08 INFO TaskSetManager: Finished task 3.0 in stage 90.0 (TID 432) in
3516 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:11:08 INFO TaskSchedulerImpl: Removed TaskSet 90.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:08 INFO DAGScheduler: ShuffleMapStage 90 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 194.261 s
25/02/06 06:11:08 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:08 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 94, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93,
ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 95, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118,
ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112, ShuffleMapStage 91,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 110)
25/02/06 06:11:08 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:08 INFO DAGScheduler: failed: Set()
25/02/06 06:11:08 INFO ShufflePartitionsUtil: For shuffle(88, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:08 INFO TaskSetManager: Starting task 0.0 in stage 94.0 (TID 454)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:08 INFO TaskSetManager: Finished task 0.0 in stage 89.0 (TID 434) in
3590 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:08 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.246.72:43231 (size: 18.4 KiB, free: 18.4 GiB)
25/02/06 06:11:08 INFO DAGScheduler: Registering RDD 1146 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 176
25/02/06 06:11:08 INFO DAGScheduler: Got map stage job 179 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:08 INFO DAGScheduler: Final stage: ShuffleMapStage 266
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:08 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 265)
25/02/06 06:11:08 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:08 INFO DAGScheduler: Submitting ShuffleMapStage 266
(MapPartitionsRDD[1146] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:08 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 266 (MapPartitionsRDD[1146] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:08 INFO TaskSchedulerImpl: Adding task set 266.0 with 9 tasks
resource profile 0
25/02/06 06:11:08 INFO TaskSetManager: TaskSet 266.0 using PreferredLocationsV1
25/02/06 06:11:08 INFO FairSchedulableBuilder: Added task set TaskSet_266.0 tasks
to pool 5400065285585087590
25/02/06 06:11:08 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098832) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098832)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:09 INFO TaskSetManager: Starting task 1.0 in stage 94.0 (TID 455)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:09 INFO TaskSetManager: Finished task 1.0 in stage 89.0 (TID 435) in
3599 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:09 INFO TaskSetManager: Starting task 2.0 in stage 94.0 (TID 456)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:09 INFO TaskSetManager: Finished task 1.0 in stage 91.0 (TID 440) in
3686 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:11:09 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.183.249.197:40023 (size: 18.4 KiB, free: 18.4 GiB)
25/02/06 06:11:10 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:10 INFO TaskSetManager: Starting task 3.0 in stage 94.0 (TID 457)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:10 INFO TaskSetManager: Finished task 2.0 in stage 91.0 (TID 441) in
3543 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:11:10 INFO TaskSetManager: Starting task 4.0 in stage 94.0 (TID 458)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:10 INFO TaskSetManager: Finished task 2.0 in stage 92.0 (TID 446) in
3566 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:10 INFO TaskSetManager: Starting task 0.0 in stage 95.0 (TID 459)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:10 INFO TaskSetManager: Finished task 3.0 in stage 92.0 (TID 447) in
3588 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:10 INFO TaskSetManager: Starting task 1.0 in stage 95.0 (TID 460)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:10 INFO TaskSetManager: Finished task 3.0 in stage 93.0 (TID 452) in
2871 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:11:11 INFO TaskSetManager: Starting task 2.0 in stage 95.0 (TID 461)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:11 INFO TaskSetManager: Finished task 4.0 in stage 93.0 (TID 453) in
3411 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:11:12 INFO TaskSetManager: Starting task 3.0 in stage 95.0 (TID 462)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:12 INFO TaskSetManager: Finished task 0.0 in stage 94.0 (TID 454) in
3265 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:11:12 INFO TaskSetManager: Starting task 4.0 in stage 95.0 (TID 463)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:12 INFO TaskSetManager: Finished task 1.0 in stage 94.0 (TID 455) in
3476 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:12 INFO TaskSetManager: Starting task 0.0 in stage 98.0 (TID 464)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:12 INFO TaskSetManager: Finished task 2.0 in stage 94.0 (TID 456) in
3247 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:11:12 INFO MemoryStore: Block broadcast_2 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:12 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:12 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:12 INFO SparkContext: Created broadcast 2 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:12 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:12 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 33 to 10.183.229.123:55848
25/02/06 06:11:13 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:13 INFO TaskSetManager: Starting task 1.0 in stage 98.0 (TID 465)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:13 INFO TaskSetManager: Finished task 3.0 in stage 94.0 (TID 457) in
3201 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:11:13 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:13 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 33 to 10.183.232.45:60256
25/02/06 06:11:13 INFO TaskSetManager: Starting task 2.0 in stage 98.0 (TID 466)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:13 INFO TaskSetManager: Finished task 0.0 in stage 98.0 (TID 464) in
1145 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:13 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:13 INFO TaskSetManager: Starting task 3.0 in stage 98.0 (TID 467)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:13 INFO TaskSetManager: Finished task 4.0 in stage 94.0 (TID 458) in
3399 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:11:13 INFO TaskSchedulerImpl: Removed TaskSet 94.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:13 INFO DAGScheduler: ShuffleMapStage 94 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 200.138 s
25/02/06 06:11:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:13 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 98, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252,
ShuffleMapStage 100, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 140,
ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244,
ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174,
ShuffleMapStage 124, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 264, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 110)
25/02/06 06:11:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:13 INFO DAGScheduler: failed: Set()
25/02/06 06:11:14 INFO TaskSetManager: Starting task 4.0 in stage 98.0 (TID 468)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Finished task 0.0 in stage 95.0 (TID 459) in
3457 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:11:14 INFO ShufflePartitionsUtil: For shuffle(85, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:14 INFO TaskSetManager: Starting task 5.0 in stage 98.0 (TID 469)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Finished task 1.0 in stage 95.0 (TID 460) in
3633 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:11:14 INFO TaskSetManager: Finished task 1.0 in stage 98.0 (TID 465) in
1105 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:14 INFO TaskSetManager: Starting task 6.0 in stage 98.0 (TID 470)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Finished task 3.0 in stage 98.0 (TID 467) in
807 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:14 INFO TaskSetManager: Finished task 2.0 in stage 95.0 (TID 461) in
3314 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:11:14 INFO TaskSetManager: Starting task 7.0 in stage 98.0 (TID 471)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Starting task 8.0 in stage 98.0 (TID 472)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Starting task 0.0 in stage 100.0 (TID 473)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Finished task 2.0 in stage 98.0 (TID 466) in
897 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:14 INFO MemoryStore: Block broadcast_3 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:14 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:14 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:14 INFO SparkContext: Created broadcast 3 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:14 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:14 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 10 to 10.183.229.123:55848
25/02/06 06:11:14 INFO TaskSetManager: Starting task 1.0 in stage 100.0 (TID 474)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:14 INFO TaskSetManager: Finished task 4.0 in stage 98.0 (TID 468) in
788 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:15 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099810) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099810)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:15 INFO DAGScheduler: Registering RDD 1154 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 177
25/02/06 06:11:15 INFO DAGScheduler: Got map stage job 180 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:15 INFO DAGScheduler: Final stage: ShuffleMapStage 268
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:15 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
267, ShuffleMapStage 97)
25/02/06 06:11:15 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:15 INFO DAGScheduler: Submitting ShuffleMapStage 268
(MapPartitionsRDD[1154] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:15 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 268 (MapPartitionsRDD[1154] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:15 INFO TaskSchedulerImpl: Adding task set 268.0 with 9 tasks
resource profile 0
25/02/06 06:11:15 INFO TaskSetManager: TaskSet 268.0 using PreferredLocationsV1
25/02/06 06:11:15 INFO FairSchedulableBuilder: Added task set TaskSet_268.0 tasks
to pool 5400065285585087590
25/02/06 06:11:15 INFO TaskSetManager: Starting task 2.0 in stage 100.0 (TID 475)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 5.0 in stage 98.0 (TID 469) in
803 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:15 INFO TaskSetManager: Starting task 3.0 in stage 100.0 (TID 476)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 8.0 in stage 98.0 (TID 472) in
517 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:15 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:15 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 10 to 10.183.232.45:60256
25/02/06 06:11:15 INFO TaskSetManager: Starting task 4.0 in stage 100.0 (TID 477)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 6.0 in stage 98.0 (TID 470) in
879 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:15 INFO TaskSetManager: Starting task 5.0 in stage 100.0 (TID 478)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 0.0 in stage 100.0 (TID 473)
in 974 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:15 INFO TaskSetManager: Starting task 6.0 in stage 100.0 (TID 479)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 1.0 in stage 100.0 (TID 474)
in 955 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:15 INFO TaskSetManager: Starting task 7.0 in stage 100.0 (TID 480)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 3.0 in stage 95.0 (TID 462) in
3786 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:11:15 INFO TaskSetManager: Starting task 8.0 in stage 100.0 (TID 481)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 7.0 in stage 98.0 (TID 471) in
1200 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:15 INFO TaskSchedulerImpl: Removed TaskSet 98.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:15 INFO DAGScheduler: ShuffleMapStage 98 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 112.159 s
25/02/06 06:11:15 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:15 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248,
ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 95,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 89, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 264, ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 110)
25/02/06 06:11:15 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:15 INFO DAGScheduler: failed: Set()
25/02/06 06:11:15 INFO TaskSetManager: Starting task 0.0 in stage 102.0 (TID 482)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:15 INFO TaskSetManager: Finished task 4.0 in stage 95.0 (TID 463) in
3501 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:11:15 INFO TaskSchedulerImpl: Removed TaskSet 95.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:15 INFO DAGScheduler: ShuffleMapStage 95 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 202.157 s
25/02/06 06:11:15 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:15 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 100,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248,
ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 192, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134,
ShuffleMapStage 92, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122,
ShuffleMapStage 224, ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:15 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:15 INFO DAGScheduler: failed: Set()
25/02/06 06:11:15 INFO MemoryStore: Block broadcast_4 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:15 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:15 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:15 INFO SparkContext: Created broadcast 4 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:16 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 36 to 10.183.229.123:55848
25/02/06 06:11:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:16 INFO TaskSetManager: Starting task 1.0 in stage 102.0 (TID 483)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 8.0 in stage 100.0 (TID 481)
in 295 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:16 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:16 INFO TaskSetManager: Starting task 2.0 in stage 102.0 (TID 484)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 2.0 in stage 100.0 (TID 475)
in 1067 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 36 to 10.183.232.45:60256
25/02/06 06:11:16 INFO ShufflePartitionsUtil: For shuffle(102), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:16 INFO TaskSetManager: Starting task 3.0 in stage 102.0 (TID 485)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 3.0 in stage 100.0 (TID 476)
in 1152 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:16 INFO TaskSetManager: Starting task 4.0 in stage 102.0 (TID 486)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 4.0 in stage 100.0 (TID 477)
in 1132 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:16 INFO TaskSetManager: Starting task 5.0 in stage 102.0 (TID 487)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 6.0 in stage 100.0 (TID 479)
in 1012 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:16 INFO TaskSetManager: Starting task 6.0 in stage 102.0 (TID 488)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 5.0 in stage 100.0 (TID 478)
in 1093 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:16 INFO TaskSetManager: Finished task 0.0 in stage 102.0 (TID 482)
in 898 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:16 INFO TaskSetManager: Starting task 7.0 in stage 102.0 (TID 489)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Starting task 8.0 in stage 102.0 (TID 490)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:16 INFO TaskSetManager: Finished task 7.0 in stage 100.0 (TID 480)
in 1091 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:16 INFO TaskSchedulerImpl: Removed TaskSet 100.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:16 INFO DAGScheduler: ShuffleMapStage 100 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 113.143 s
25/02/06 06:11:16 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:16 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 102, ShuffleMapStage 204, ShuffleMapStage 154,
ShuffleMapStage 256, ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156,
ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250,
ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ResultStage 190, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:16 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:16 INFO DAGScheduler: failed: Set()
25/02/06 06:11:16 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097775) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097775)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:17 INFO CodeGenerator: Code generated in 69.931087 ms
25/02/06 06:11:17 INFO DAGScheduler: Registering RDD 1159
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 178
25/02/06 06:11:17 INFO DAGScheduler: Got map stage job 181
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:17 INFO DAGScheduler: Final stage: ShuffleMapStage 271
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:17 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
270)
25/02/06 06:11:17 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:17 INFO DAGScheduler: Submitting ShuffleMapStage 271
(MapPartitionsRDD[1159] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:17 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 271 (MapPartitionsRDD[1159] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:17 INFO TaskSchedulerImpl: Adding task set 271.0 with 6 tasks
resource profile 0
25/02/06 06:11:17 INFO TaskSetManager: TaskSet 271.0 using PreferredLocationsV1
25/02/06 06:11:17 INFO FairSchedulableBuilder: Added task set TaskSet_271.0 tasks
to pool 5400065285585087590
25/02/06 06:11:17 INFO TaskSetManager: Starting task 0.0 in stage 104.0 (TID 491)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:17 INFO TaskSetManager: Finished task 2.0 in stage 102.0 (TID 484)
in 881 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:17 INFO MemoryStore: Block broadcast_5 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:17 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:17 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:17 INFO SparkContext: Created broadcast 5 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:17 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:17 INFO TaskSetManager: Finished task 1.0 in stage 102.0 (TID 483)
in 934 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:17 INFO TaskSetManager: Starting task 1.0 in stage 104.0 (TID 492)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 20 to 10.183.229.123:55848
25/02/06 06:11:17 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:17 INFO TaskSetManager: Finished task 3.0 in stage 102.0 (TID 485)
in 778 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:17 INFO TaskSetManager: Starting task 2.0 in stage 104.0 (TID 493)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 20 to 10.183.232.45:60256
25/02/06 06:11:17 INFO TaskSetManager: Starting task 3.0 in stage 104.0 (TID 494)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:17 INFO TaskSetManager: Finished task 8.0 in stage 102.0 (TID 490)
in 273 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:17 INFO ShufflePartitionsUtil: For shuffle(92, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:17 INFO TaskSetManager: Starting task 4.0 in stage 104.0 (TID 495)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:17 INFO TaskSetManager: Finished task 6.0 in stage 102.0 (TID 488)
in 815 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:17 INFO TaskSetManager: Starting task 5.0 in stage 104.0 (TID 496)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:17 INFO TaskSetManager: Finished task 5.0 in stage 102.0 (TID 487)
in 890 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:17 INFO TaskSetManager: Finished task 4.0 in stage 102.0 (TID 486)
in 1019 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:17 INFO TaskSetManager: Starting task 6.0 in stage 104.0 (TID 497)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Starting task 7.0 in stage 104.0 (TID 498)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Starting task 8.0 in stage 104.0 (TID 499)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 7.0 in stage 102.0 (TID 489)
in 1144 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:18 INFO TaskSchedulerImpl: Removed TaskSet 102.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:18 INFO TaskSetManager: Finished task 0.0 in stage 104.0 (TID 491)
in 851 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:18 INFO DAGScheduler: ShuffleMapStage 102 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 114.208 s
25/02/06 06:11:18 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:18 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 104, ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 106, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 150, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:18 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:18 INFO DAGScheduler: failed: Set()
25/02/06 06:11:18 INFO TaskSetManager: Finished task 2.0 in stage 104.0 (TID 493)
in 890 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:18 INFO TaskSetManager: Starting task 0.0 in stage 106.0 (TID 500)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:18 INFO MemoryStore: Block broadcast_6 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:18 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:18 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:18 INFO SparkContext: Created broadcast 6 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:18 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 28 to 10.183.232.45:60256
25/02/06 06:11:18 INFO DAGScheduler: Registering RDD 1167 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 179
25/02/06 06:11:18 INFO DAGScheduler: Got map stage job 182 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:18 INFO DAGScheduler: Final stage: ShuffleMapStage 273
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:18 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
272, ShuffleMapStage 97)
25/02/06 06:11:18 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:18 INFO DAGScheduler: Submitting ShuffleMapStage 273
(MapPartitionsRDD[1167] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:18 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 273 (MapPartitionsRDD[1167] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:18 INFO TaskSchedulerImpl: Adding task set 273.0 with 9 tasks
resource profile 0
25/02/06 06:11:18 INFO TaskSetManager: TaskSet 273.0 using PreferredLocationsV1
25/02/06 06:11:18 INFO FairSchedulableBuilder: Added task set TaskSet_273.0 tasks
to pool 5400065285585087590
25/02/06 06:11:18 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098872) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098872)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:18 INFO TaskSetManager: Starting task 1.0 in stage 106.0 (TID 501)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Starting task 2.0 in stage 106.0 (TID 502)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 1.0 in stage 104.0 (TID 492)
in 1029 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:18 INFO TaskSetManager: Finished task 3.0 in stage 104.0 (TID 494)
in 994 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:18 INFO TaskSetManager: Starting task 3.0 in stage 106.0 (TID 503)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 8.0 in stage 104.0 (TID 499)
in 367 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:18 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 28 to 10.183.229.123:55848
25/02/06 06:11:18 INFO TaskSetManager: Starting task 4.0 in stage 106.0 (TID 504)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 4.0 in stage 104.0 (TID 495)
in 899 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:18 INFO ShufflePartitionsUtil: For shuffle(101), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:18 INFO ShufflePartitionsUtil: For shuffle(103), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:18 INFO TaskSetManager: Starting task 5.0 in stage 106.0 (TID 505)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 6.0 in stage 104.0 (TID 497)
in 855 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:18 INFO TaskSetManager: Starting task 6.0 in stage 106.0 (TID 506)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 5.0 in stage 104.0 (TID 496)
in 1011 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:18 INFO TaskSetManager: Starting task 7.0 in stage 106.0 (TID 507)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 0.0 in stage 106.0 (TID 500)
in 805 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:18 INFO TaskSetManager: Starting task 8.0 in stage 106.0 (TID 508)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Starting task 0.0 in stage 108.0 (TID 509)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:18 INFO TaskSetManager: Finished task 1.0 in stage 106.0 (TID 501)
in 730 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:18 INFO MemoryStore: Block broadcast_7 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:18 INFO TaskSetManager: Finished task 7.0 in stage 104.0 (TID 498)
in 929 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:18 INFO TaskSchedulerImpl: Removed TaskSet 104.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:18 INFO DAGScheduler: ShuffleMapStage 104 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 115.132 s
25/02/06 06:11:18 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:18 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 106, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 124, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:18 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:18 INFO DAGScheduler: failed: Set()
25/02/06 06:11:18 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:18 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:18 INFO SparkContext: Created broadcast 7 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:18 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 17 to 10.183.232.45:60256
25/02/06 06:11:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:19 INFO TaskSetManager: Starting task 1.0 in stage 108.0 (TID 510)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 3.0 in stage 106.0 (TID 503)
in 793 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:19 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097784) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097784)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:19 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:19 INFO TaskSetManager: Finished task 2.0 in stage 106.0 (TID 502)
in 998 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 2.0 in stage 108.0 (TID 511)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 17 to 10.183.229.123:55848
25/02/06 06:11:19 INFO CodeGenerator: Code generated in 52.482209 ms
25/02/06 06:11:19 INFO TaskSetManager: Starting task 3.0 in stage 108.0 (TID 512)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 8.0 in stage 106.0 (TID 508)
in 298 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:19 INFO DAGScheduler: Registering RDD 1172
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 180
25/02/06 06:11:19 INFO DAGScheduler: Got map stage job 183
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:19 INFO DAGScheduler: Final stage: ShuffleMapStage 276
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:19 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
275)
25/02/06 06:11:19 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:19 INFO DAGScheduler: Submitting ShuffleMapStage 276
(MapPartitionsRDD[1172] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:19 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 276 (MapPartitionsRDD[1172] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:19 INFO TaskSchedulerImpl: Adding task set 276.0 with 6 tasks
resource profile 0
25/02/06 06:11:19 INFO TaskSetManager: TaskSet 276.0 using PreferredLocationsV1
25/02/06 06:11:19 INFO FairSchedulableBuilder: Added task set TaskSet_276.0 tasks
to pool 5400065285585087590
25/02/06 06:11:19 INFO CodeGenerator: Code generated in 92.341681 ms
25/02/06 06:11:19 INFO DAGScheduler: Registering RDD 1177
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 181
25/02/06 06:11:19 INFO DAGScheduler: Got map stage job 184
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:19 INFO DAGScheduler: Final stage: ShuffleMapStage 279
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:19 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
278)
25/02/06 06:11:19 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:19 INFO DAGScheduler: Submitting ShuffleMapStage 279
(MapPartitionsRDD[1177] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:19 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 279 (MapPartitionsRDD[1177] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:19 INFO TaskSchedulerImpl: Adding task set 279.0 with 5 tasks
resource profile 0
25/02/06 06:11:19 INFO TaskSetManager: TaskSet 279.0 using PreferredLocationsV1
25/02/06 06:11:19 INFO FairSchedulableBuilder: Added task set TaskSet_279.0 tasks
to pool 5400065285585087590
25/02/06 06:11:19 INFO TaskSetManager: Starting task 4.0 in stage 108.0 (TID 513)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 5.0 in stage 106.0 (TID 505)
in 753 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 5.0 in stage 108.0 (TID 514)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 4.0 in stage 106.0 (TID 504)
in 889 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 6.0 in stage 108.0 (TID 515)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 6.0 in stage 106.0 (TID 506)
in 813 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 7.0 in stage 108.0 (TID 516)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 7.0 in stage 106.0 (TID 507)
in 697 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:19 INFO TaskSchedulerImpl: Removed TaskSet 106.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:19 INFO DAGScheduler: ShuffleMapStage 106 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 115.759 s
25/02/06 06:11:19 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:19 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 89,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264,
ShuffleMapStage 112, ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:19 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:19 INFO DAGScheduler: failed: Set()
25/02/06 06:11:19 INFO ShufflePartitionsUtil: For shuffle(106), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:19 INFO TaskSetManager: Starting task 8.0 in stage 108.0 (TID 517)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 0.0 in stage 91.0 (TID 439) in
14403 ms on 10.183.233.191 (executor 5) (3/5)
25/02/06 06:11:19 INFO TaskSetManager: Finished task 2.0 in stage 89.0 (TID 436) in
14404 ms on 10.183.233.191 (executor 5) (3/5)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 0.0 in stage 110.0 (TID 518)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Starting task 1.0 in stage 110.0 (TID 519)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 3.0 in stage 89.0 (TID 437) in
14408 ms on 10.183.233.191 (executor 5) (4/5)
25/02/06 06:11:19 INFO MemoryStore: Block broadcast_8 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:19 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in
memory (estimated size 38.6 KiB, free 20.2 GiB)
25/02/06 06:11:19 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on
10.183.232.44:37705 (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:11:19 INFO SparkContext: Created broadcast 8 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:19 INFO TaskSetManager: Starting task 2.0 in stage 110.0 (TID 520)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 0.0 in stage 108.0 (TID 509)
in 944 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:19 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on
10.183.232.45:45579 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:19 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:19 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on
10.183.233.191:42923 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:19 INFO TaskSetManager: Starting task 3.0 in stage 110.0 (TID 521)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:19 INFO TaskSetManager: Finished task 4.0 in stage 89.0 (TID 438) in
14450 ms on 10.183.233.191 (executor 5) (5/5)
25/02/06 06:11:19 INFO TaskSchedulerImpl: Removed TaskSet 89.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:19 INFO DAGScheduler: ShuffleMapStage 89 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 206.154 s
25/02/06 06:11:19 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:19 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 91, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:19 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:19 INFO DAGScheduler: failed: Set()
25/02/06 06:11:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 14 to 10.183.232.45:60256
25/02/06 06:11:20 INFO TaskSetManager: Starting task 4.0 in stage 110.0 (TID 522)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Starting task 5.0 in stage 110.0 (TID 523)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 17 to 10.183.233.191:55538
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 14 to 10.183.233.191:55538
25/02/06 06:11:20 INFO TaskSetManager: Finished task 0.0 in stage 92.0 (TID 444) in
13148 ms on 10.183.246.72 (executor 6) (3/5)
25/02/06 06:11:20 INFO TaskSetManager: Finished task 4.0 in stage 91.0 (TID 443) in
13150 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 6.0 in stage 110.0 (TID 524)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 3.0 in stage 91.0 (TID 442) in
13153 ms on 10.183.246.72 (executor 6) (5/5)
25/02/06 06:11:20 INFO TaskSchedulerImpl: Removed TaskSet 91.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:20 INFO DAGScheduler: ShuffleMapStage 91 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 206.275 s
25/02/06 06:11:20 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:20 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 108, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 110)
25/02/06 06:11:20 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:20 INFO DAGScheduler: failed: Set()
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on
10.183.246.72:43231 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 7.0 in stage 110.0 (TID 525)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 1.0 in stage 92.0 (TID 445) in
13186 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 8.0 in stage 110.0 (TID 526)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 2.0 in stage 108.0 (TID 511)
in 891 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 0.0 in stage 112.0 (TID 527)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 3.0 in stage 108.0 (TID 512)
in 876 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:20 INFO MemoryStore: Block broadcast_9 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:20 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:20 INFO SparkContext: Created broadcast 9 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 1.0 in stage 112.0 (TID 528)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 1.0 in stage 108.0 (TID 510)
in 964 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 16 to 10.183.229.123:55848
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 14 to 10.183.246.72:39896
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:20 INFO TaskSetManager: Finished task 4.0 in stage 108.0 (TID 513)
in 974 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 2.0 in stage 112.0 (TID 529)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 8.0 in stage 110.0 (TID 526)
in 253 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 3.0 in stage 112.0 (TID 530)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 16 to 10.183.232.45:60256
25/02/06 06:11:20 INFO TaskSetManager: Starting task 4.0 in stage 112.0 (TID 531)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 5.0 in stage 108.0 (TID 514)
in 959 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 5.0 in stage 112.0 (TID 532)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 6.0 in stage 108.0 (TID 515)
in 929 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:20 INFO TaskSetManager: Finished task 7.0 in stage 108.0 (TID 516)
in 934 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:20 INFO TaskSetManager: Starting task 6.0 in stage 112.0 (TID 533)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Starting task 7.0 in stage 112.0 (TID 534)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Starting task 8.0 in stage 112.0 (TID 535)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:20 INFO TaskSetManager: Finished task 2.0 in stage 110.0 (TID 520)
in 909 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:20 INFO TaskSetManager: Finished task 8.0 in stage 108.0 (TID 517)
in 936 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:20 INFO TaskSchedulerImpl: Removed TaskSet 108.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:20 INFO DAGScheduler: ShuffleMapStage 108 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 116.952 s
25/02/06 06:11:20 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:20 INFO DAGScheduler: running: Set(ShuffleMapStage 254,
ShuffleMapStage 212, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92,
ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 110)
25/02/06 06:11:20 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:20 INFO DAGScheduler: failed: Set()
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 16 to 10.183.233.191:55538
25/02/06 06:11:20 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099861) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099861)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:20 INFO TaskSetManager: Starting task 0.0 in stage 114.0 (TID 536)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:20 INFO MemoryStore: Block broadcast_10 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:20 INFO TaskSetManager: Finished task 0.0 in stage 112.0 (TID 527)
in 840 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:20 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:20 INFO SparkContext: Created broadcast 10 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:20 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 35 to 10.183.229.123:55848
25/02/06 06:11:20 INFO CodeGenerator: Code generated in 32.962823 ms
25/02/06 06:11:21 INFO DAGScheduler: Registering RDD 1182
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 182
25/02/06 06:11:21 INFO DAGScheduler: Got map stage job 185
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:21 INFO DAGScheduler: Final stage: ShuffleMapStage 282
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:21 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
281)
25/02/06 06:11:21 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:21 INFO DAGScheduler: Submitting ShuffleMapStage 282
(MapPartitionsRDD[1182] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:21 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 282 (MapPartitionsRDD[1182] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:21 INFO TaskSchedulerImpl: Adding task set 282.0 with 5 tasks
resource profile 0
25/02/06 06:11:21 INFO TaskSetManager: TaskSet 282.0 using PreferredLocationsV1
25/02/06 06:11:21 INFO FairSchedulableBuilder: Added task set TaskSet_282.0 tasks
to pool 5400065285585087590
25/02/06 06:11:21 INFO TaskSetManager: Starting task 1.0 in stage 114.0 (TID 537)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 8.0 in stage 112.0 (TID 535)
in 272 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 35 to 10.183.232.45:60256
25/02/06 06:11:21 INFO TaskSetManager: Starting task 2.0 in stage 114.0 (TID 538)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 3.0 in stage 112.0 (TID 530)
in 742 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 3.0 in stage 114.0 (TID 539)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 1.0 in stage 112.0 (TID 528)
in 1010 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:21 INFO TaskSetManager: Finished task 2.0 in stage 112.0 (TID 529)
in 822 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 4.0 in stage 114.0 (TID 540)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Starting task 5.0 in stage 114.0 (TID 541)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 5.0 in stage 112.0 (TID 532)
in 762 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:21 INFO TaskSetManager: Finished task 4.0 in stage 112.0 (TID 531)
in 809 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 6.0 in stage 114.0 (TID 542)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Starting task 7.0 in stage 114.0 (TID 543)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 3.0 in stage 110.0 (TID 521)
in 1400 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO TaskSetManager: Finished task 7.0 in stage 110.0 (TID 525)
in 1251 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 8.0 in stage 114.0 (TID 544)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 4.0 in stage 110.0 (TID 522)
in 1293 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 35 to 10.183.233.191:55538
25/02/06 06:11:21 INFO TaskSetManager: Starting task 0.0 in stage 116.0 (TID 545)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:21 INFO MemoryStore: Block broadcast_11 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:21 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:21 INFO SparkContext: Created broadcast 11 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 6 to 10.183.246.72:39896
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 35 to 10.183.246.72:39896
25/02/06 06:11:21 INFO TaskSetManager: Starting task 1.0 in stage 116.0 (TID 546)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 6.0 in stage 110.0 (TID 524)
in 1367 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 2.0 in stage 116.0 (TID 547)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 6.0 in stage 112.0 (TID 533)
in 942 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:21 INFO ShufflePartitionsUtil: For shuffle(108), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:21 INFO ShufflePartitionsUtil: For shuffle(107), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:21 INFO TaskSetManager: Finished task 0.0 in stage 110.0 (TID 518)
in 1635 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 3.0 in stage 116.0 (TID 548)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO ShufflePartitionsUtil: For shuffle(87, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 6 to 10.183.233.191:55538
25/02/06 06:11:21 INFO ShufflePartitionsUtil: For shuffle(89, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 6 to 10.183.232.45:60256
25/02/06 06:11:21 INFO TaskSetManager: Starting task 4.0 in stage 116.0 (TID 549)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 1.0 in stage 110.0 (TID 519)
in 1808 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 5.0 in stage 116.0 (TID 550)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 5.0 in stage 110.0 (TID 523)
in 1632 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:21 INFO TaskSchedulerImpl: Removed TaskSet 110.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:21 INFO DAGScheduler: ShuffleMapStage 110 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 117.828 s
25/02/06 06:11:21 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:21 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 112,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:21 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:21 INFO DAGScheduler: failed: Set()
25/02/06 06:11:21 INFO TaskSetManager: Starting task 6.0 in stage 116.0 (TID 551)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 1.0 in stage 114.0 (TID 537)
in 724 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 7.0 in stage 116.0 (TID 552)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 0.0 in stage 114.0 (TID 536)
in 935 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 8.0 in stage 116.0 (TID 553)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 2.0 in stage 114.0 (TID 538)
in 806 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 6 to 10.183.229.123:55848
25/02/06 06:11:21 INFO TaskSetManager: Starting task 0.0 in stage 124.0 (TID 554)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:21 INFO TaskSetManager: Finished task 7.0 in stage 112.0 (TID 534)
in 1139 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:21 INFO TaskSchedulerImpl: Removed TaskSet 112.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:21 INFO DAGScheduler: ShuffleMapStage 112 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 118.070 s
25/02/06 06:11:21 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:21 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 93, ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:21 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:21 INFO DAGScheduler: failed: Set()
25/02/06 06:11:21 INFO MemoryStore: Block broadcast_12 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:21 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:21 INFO SparkContext: Created broadcast 12 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:21 INFO TaskSetManager: Finished task 3.0 in stage 114.0 (TID 539)
in 801 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:21 INFO TaskSetManager: Starting task 1.0 in stage 124.0 (TID 555)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 45 to 10.183.233.191:55538
25/02/06 06:11:21 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 45 to 10.183.229.123:55848
25/02/06 06:11:22 INFO TaskSetManager: Starting task 2.0 in stage 124.0 (TID 556)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 8.0 in stage 114.0 (TID 544)
in 728 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:22 INFO TaskSetManager: Starting task 3.0 in stage 124.0 (TID 557)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Starting task 4.0 in stage 124.0 (TID 558)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 0.0 in stage 93.0 (TID 449) in
14538 ms on 10.183.249.197 (executor 7) (3/5)
25/02/06 06:11:22 INFO TaskSetManager: Finished task 2.0 in stage 93.0 (TID 451) in
14538 ms on 10.183.249.197 (executor 7) (4/5)
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 5.0 in stage 124.0 (TID 559)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 1.0 in stage 93.0 (TID 450) in
14542 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:11:22 INFO TaskSchedulerImpl: Removed TaskSet 93.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:22 INFO DAGScheduler: ShuffleMapStage 93 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 208.306 s
25/02/06 06:11:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:22 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 92, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:22 INFO DAGScheduler: failed: Set()
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 45 to 10.183.246.72:39896
25/02/06 06:11:22 INFO DAGScheduler: Registering RDD 1190 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 183
25/02/06 06:11:22 INFO DAGScheduler: Got map stage job 186 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:22 INFO DAGScheduler: Final stage: ShuffleMapStage 284
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 283)
25/02/06 06:11:22 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:22 INFO DAGScheduler: Submitting ShuffleMapStage 284
(MapPartitionsRDD[1190] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:22 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098840) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098840)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:22 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 284 (MapPartitionsRDD[1190] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:22 INFO TaskSchedulerImpl: Adding task set 284.0 with 9 tasks
resource profile 0
25/02/06 06:11:22 INFO TaskSetManager: TaskSet 284.0 using PreferredLocationsV1
25/02/06 06:11:22 INFO FairSchedulableBuilder: Added task set TaskSet_284.0 tasks
to pool 5400065285585087590
25/02/06 06:11:22 INFO TaskSetManager: Finished task 4.0 in stage 92.0 (TID 448) in
14585 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:11:22 INFO TaskSchedulerImpl: Removed TaskSet 92.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:22 INFO DAGScheduler: ShuffleMapStage 92 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 208.357 s
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:22 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 142,
ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 284, ShuffleMapStage 132, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 174, ShuffleMapStage 276, ShuffleMapStage 124, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 116, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 114, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:22 INFO DAGScheduler: failed: Set()
25/02/06 06:11:22 INFO TaskSetManager: Starting task 6.0 in stage 124.0 (TID 560)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:22 INFO DAGScheduler: Registering RDD 1198 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 184
25/02/06 06:11:22 INFO DAGScheduler: Got map stage job 187 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:22 INFO DAGScheduler: Final stage: ShuffleMapStage 286
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
285, ShuffleMapStage 97)
25/02/06 06:11:22 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:22 INFO DAGScheduler: Submitting ShuffleMapStage 286
(MapPartitionsRDD[1198] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:22 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 286 (MapPartitionsRDD[1198] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:22 INFO TaskSchedulerImpl: Adding task set 286.0 with 9 tasks
resource profile 0
25/02/06 06:11:22 INFO TaskSetManager: TaskSet 286.0 using PreferredLocationsV1
25/02/06 06:11:22 INFO FairSchedulableBuilder: Added task set TaskSet_286.0 tasks
to pool 5400065285585087590
25/02/06 06:11:22 INFO TaskSetManager: Finished task 6.0 in stage 114.0 (TID 542)
in 934 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:22 INFO TaskSetManager: Finished task 8.0 in stage 116.0 (TID 553)
in 257 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 7.0 in stage 124.0 (TID 561)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Starting task 8.0 in stage 124.0 (TID 562)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 4.0 in stage 114.0 (TID 540)
in 987 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 0.0 in stage 118.0 (TID 563)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:22 INFO MemoryStore: Block broadcast_13 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:22 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:22 INFO SparkContext: Created broadcast 13 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:22 INFO CodeGenerator: Code generated in 64.950437 ms
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO CodeGenerator: Code generated in 87.276365 ms
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 19 to 10.183.232.45:60256
25/02/06 06:11:22 INFO DAGScheduler: Registering RDD 1203
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 185
25/02/06 06:11:22 INFO DAGScheduler: Got map stage job 188
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:22 INFO DAGScheduler: Final stage: ShuffleMapStage 289
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
288)
25/02/06 06:11:22 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:22 INFO DAGScheduler: Submitting ShuffleMapStage 289
(MapPartitionsRDD[1203] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:22 INFO DAGScheduler: Registering RDD 1208
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 186
25/02/06 06:11:22 INFO DAGScheduler: Got map stage job 189
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:22 INFO DAGScheduler: Final stage: ShuffleMapStage 292
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
291)
25/02/06 06:11:22 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:22 INFO DAGScheduler: Submitting ShuffleMapStage 292
(MapPartitionsRDD[1208] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:22 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 289 (MapPartitionsRDD[1203] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:22 INFO TaskSchedulerImpl: Adding task set 289.0 with 5 tasks
resource profile 0
25/02/06 06:11:22 INFO TaskSetManager: TaskSet 289.0 using PreferredLocationsV1
25/02/06 06:11:22 INFO FairSchedulableBuilder: Added task set TaskSet_289.0 tasks
to pool 5400065285585087590
25/02/06 06:11:22 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 292 (MapPartitionsRDD[1208] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:22 INFO TaskSchedulerImpl: Adding task set 292.0 with 6 tasks
resource profile 0
25/02/06 06:11:22 INFO TaskSetManager: TaskSet 292.0 using PreferredLocationsV1
25/02/06 06:11:22 INFO FairSchedulableBuilder: Added task set TaskSet_292.0 tasks
to pool 5400065285585087590
25/02/06 06:11:22 INFO TaskSetManager: Finished task 5.0 in stage 114.0 (TID 541)
in 1036 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 1.0 in stage 118.0 (TID 564)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 45 to 10.183.232.45:60256
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 19 to 10.183.229.123:55848
25/02/06 06:11:22 INFO TaskSetManager: Finished task 2.0 in stage 116.0 (TID 547)
in 858 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 2.0 in stage 118.0 (TID 565)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 45 to 10.183.249.197:36834
25/02/06 06:11:22 INFO TaskSetManager: Starting task 3.0 in stage 118.0 (TID 566)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 7.0 in stage 114.0 (TID 543)
in 1130 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:22 INFO TaskSchedulerImpl: Removed TaskSet 114.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:22 INFO DAGScheduler: ShuffleMapStage 114 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 118.571 s
25/02/06 06:11:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:22 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 284, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 276,
ShuffleMapStage 124, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 116,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:22 INFO DAGScheduler: failed: Set()
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO TaskSetManager: Finished task 0.0 in stage 116.0 (TID 545)
in 1147 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 4.0 in stage 118.0 (TID 567)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 19 to 10.183.233.191:55538
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 19 to 10.183.246.72:39896
25/02/06 06:11:22 INFO TaskSetManager: Finished task 5.0 in stage 116.0 (TID 550)
in 883 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 5.0 in stage 118.0 (TID 568)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:22 INFO ShufflePartitionsUtil: For shuffle(99), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:22 INFO ShufflePartitionsUtil: For shuffle(98), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:22 INFO TaskSetManager: Starting task 6.0 in stage 118.0 (TID 569)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 8.0 in stage 124.0 (TID 562)
in 451 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 7.0 in stage 118.0 (TID 570)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 3.0 in stage 116.0 (TID 548)
in 1127 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:22 INFO TaskSetManager: Finished task 6.0 in stage 116.0 (TID 551)
in 845 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 8.0 in stage 118.0 (TID 571)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:22 INFO ShufflePartitionsUtil: For shuffle(91, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:22 INFO TaskSetManager: Starting task 0.0 in stage 120.0 (TID 572)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 7.0 in stage 116.0 (TID 552)
in 843 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:22 INFO MemoryStore: Block broadcast_14 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:22 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:22 INFO SparkContext: Created broadcast 14 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 24 to 10.183.229.123:55848
25/02/06 06:11:22 INFO TaskSetManager: Finished task 1.0 in stage 116.0 (TID 546)
in 1334 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 1.0 in stage 120.0 (TID 573)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 24 to 10.183.246.72:39896
25/02/06 06:11:22 INFO TaskSetManager: Finished task 4.0 in stage 116.0 (TID 549)
in 1121 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:22 INFO TaskSchedulerImpl: Removed TaskSet 116.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:22 INFO TaskSetManager: Starting task 2.0 in stage 120.0 (TID 574)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:22 INFO DAGScheduler: ShuffleMapStage 116 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 118.918 s
25/02/06 06:11:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:22 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 284, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 276,
ShuffleMapStage 124, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 118, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:22 INFO DAGScheduler: failed: Set()
25/02/06 06:11:22 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 24 to 10.183.233.191:55538
25/02/06 06:11:22 INFO TaskSetManager: Starting task 3.0 in stage 120.0 (TID 575)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 0.0 in stage 118.0 (TID 563)
in 813 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:22 INFO TaskSetManager: Starting task 4.0 in stage 120.0 (TID 576)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:22 INFO TaskSetManager: Finished task 8.0 in stage 118.0 (TID 571)
in 349 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 24 to 10.183.232.45:60256
25/02/06 06:11:23 INFO TaskSetManager: Starting task 5.0 in stage 120.0 (TID 577)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 1.0 in stage 124.0 (TID 555)
in 1079 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 0.0 in stage 124.0 (TID 554)
in 1508 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 6.0 in stage 120.0 (TID 578)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 7.0 in stage 120.0 (TID 579)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 8.0 in stage 120.0 (TID 580)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 0.0 in stage 122.0 (TID 581)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 1.0 in stage 122.0 (TID 582)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 2.0 in stage 122.0 (TID 583)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 7.0 in stage 118.0 (TID 570)
in 819 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 3.0 in stage 122.0 (TID 584)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 0.0 in stage 120.0 (TID 572)
in 720 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 7.0 in stage 124.0 (TID 561)
in 1286 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 2.0 in stage 124.0 (TID 556)
in 1380 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:23 INFO MemoryStore: Block broadcast_15 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 2.0 in stage 118.0 (TID 565)
in 1121 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 1.0 in stage 118.0 (TID 564)
in 1203 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 3.0 in stage 118.0 (TID 566)
in 1010 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 6.0 in stage 118.0 (TID 569)
in 838 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 4.0 in stage 122.0 (TID 585)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Starting task 5.0 in stage 122.0 (TID 586)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:23 INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:23 INFO SparkContext: Created broadcast 15 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 32 to 10.183.229.123:55848
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 32 to 10.183.232.45:60256
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 32 to 10.183.246.72:39896
25/02/06 06:11:23 INFO TaskSetManager: Finished task 4.0 in stage 118.0 (TID 567)
in 1002 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 6.0 in stage 122.0 (TID 587)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:11:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:11:23 INFO TaskSetManager: Starting task 7.0 in stage 122.0 (TID 588)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 3.0 in stage 120.0 (TID 575)
in 580 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 8.0 in stage 122.0 (TID 589)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 4.0 in stage 124.0 (TID 558)
in 1483 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 0.0 in stage 126.0 (TID 590)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 5.0 in stage 118.0 (TID 568)
in 1025 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:23 INFO TaskSchedulerImpl: Removed TaskSet 118.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:23 INFO MemoryStore: Block broadcast_16 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:23 INFO MemoryStore: Block broadcast_16_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:23 INFO DAGScheduler: ShuffleMapStage 118 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 119.711 s
25/02/06 06:11:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:23 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 284, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 276,
ShuffleMapStage 124, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 120, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:23 INFO DAGScheduler: failed: Set()
25/02/06 06:11:23 INFO TaskSetManager: Finished task 1.0 in stage 120.0 (TID 573)
in 843 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 1.0 in stage 126.0 (TID 591)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:23 INFO SparkContext: Created broadcast 16 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO DriverCorral: DBFS health check ok
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 15 to 10.183.246.72:39896
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 32 to 10.183.249.197:36834
25/02/06 06:11:23 INFO TaskSetManager: Starting task 2.0 in stage 126.0 (TID 592)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 5.0 in stage 120.0 (TID 577)
in 629 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO TaskSetManager: Finished task 5.0 in stage 124.0 (TID 559)
in 1562 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 3.0 in stage 126.0 (TID 593)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 15 to 10.183.229.123:55848
25/02/06 06:11:23 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:11:23 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 4.0 in stage 126.0 (TID 594)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 2.0 in stage 120.0 (TID 574)
in 923 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 15 to 10.183.249.197:36834
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 5.0 in stage 126.0 (TID 595)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 6.0 in stage 124.0 (TID 560)
in 1594 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 15 to 10.183.233.191:55538
25/02/06 06:11:23 INFO TaskSetManager: Starting task 6.0 in stage 126.0 (TID 596)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 3.0 in stage 124.0 (TID 557)
in 1689 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:23 INFO TaskSchedulerImpl: Removed TaskSet 124.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:23 INFO DAGScheduler: ShuffleMapStage 124 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 119.880 s
25/02/06 06:11:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:23 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 130,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 284, ShuffleMapStage 132,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 126, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 122, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 120,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:23 INFO DAGScheduler: failed: Set()
25/02/06 06:11:23 INFO TaskSetManager: Starting task 7.0 in stage 126.0 (TID 597)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:23 INFO TaskSetManager: Finished task 8.0 in stage 120.0 (TID 580)
in 358 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:23 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098961) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098961)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:23 INFO TaskSetManager: Finished task 4.0 in stage 120.0 (TID 576)
in 840 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:23 INFO TaskSetManager: Starting task 8.0 in stage 126.0 (TID 598)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:23 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:23 INFO DAGScheduler: Registering RDD 1216 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 187
25/02/06 06:11:23 INFO DAGScheduler: Got map stage job 190 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:23 INFO DAGScheduler: Final stage: ShuffleMapStage 294
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
293, ShuffleMapStage 97)
25/02/06 06:11:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 15 to 10.183.232.45:60256
25/02/06 06:11:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:23 INFO DAGScheduler: Submitting ShuffleMapStage 294
(MapPartitionsRDD[1216] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:23 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 294 (MapPartitionsRDD[1216] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:23 INFO TaskSchedulerImpl: Adding task set 294.0 with 9 tasks
resource profile 0
25/02/06 06:11:23 INFO TaskSetManager: TaskSet 294.0 using PreferredLocationsV1
25/02/06 06:11:23 INFO FairSchedulableBuilder: Added task set TaskSet_294.0 tasks
to pool 5400065285585087590
25/02/06 06:11:23 INFO CodeGenerator: Code generated in 62.934837 ms
25/02/06 06:11:23 INFO DAGScheduler: Registering RDD 1221
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 188
25/02/06 06:11:23 INFO DAGScheduler: Got map stage job 191
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:23 INFO DAGScheduler: Final stage: ShuffleMapStage 297
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
296)
25/02/06 06:11:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:23 INFO DAGScheduler: Submitting ShuffleMapStage 297
(MapPartitionsRDD[1221] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:23 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 297 (MapPartitionsRDD[1221] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:23 INFO TaskSchedulerImpl: Adding task set 297.0 with 6 tasks
resource profile 0
25/02/06 06:11:23 INFO TaskSetManager: TaskSet 297.0 using PreferredLocationsV1
25/02/06 06:11:23 INFO FairSchedulableBuilder: Added task set TaskSet_297.0 tasks
to pool 5400065285585087590
25/02/06 06:11:23 INFO CodeGenerator: Code generated in 86.411909 ms
25/02/06 06:11:23 INFO DAGScheduler: Registering RDD 1226
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 189
25/02/06 06:11:23 INFO DAGScheduler: Got map stage job 192
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:23 INFO DAGScheduler: Final stage: ShuffleMapStage 300
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
299)
25/02/06 06:11:23 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:23 INFO DAGScheduler: Submitting ShuffleMapStage 300
(MapPartitionsRDD[1226] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:23 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 300 (MapPartitionsRDD[1226] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:23 INFO TaskSchedulerImpl: Adding task set 300.0 with 6 tasks
resource profile 0
25/02/06 06:11:23 INFO TaskSetManager: TaskSet 300.0 using PreferredLocationsV1
25/02/06 06:11:23 INFO FairSchedulableBuilder: Added task set TaskSet_300.0 tasks
to pool 5400065285585087590
25/02/06 06:11:24 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(188,1738822282219,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@e68df6, org.apache.spark.scheduler.StageInfo@3b4538b4,
org.apache.spark.scheduler.StageInfo@3fddf9e5,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 1.028230353s.
25/02/06 06:11:24 INFO TaskSetManager: Starting task 0.0 in stage 130.0 (TID 599)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 8.0 in stage 126.0 (TID 598)
in 293 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_17 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_17_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:24 INFO SparkContext: Created broadcast 17 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:24 INFO TaskSetManager: Finished task 8.0 in stage 122.0 (TID 589)
in 550 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 1.0 in stage 130.0 (TID 600)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 13 to 10.183.232.45:60256
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 13 to 10.183.249.197:36834
25/02/06 06:11:24 INFO DriverCorral: Metastore health check ok
25/02/06 06:11:24 INFO TaskSetManager: Starting task 2.0 in stage 130.0 (TID 601)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 6.0 in stage 120.0 (TID 578)
in 796 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 3.0 in stage 130.0 (TID 602)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 4.0 in stage 122.0 (TID 585)
in 794 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 4.0 in stage 130.0 (TID 603)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 7.0 in stage 120.0 (TID 579)
in 820 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:24 INFO TaskSchedulerImpl: Removed TaskSet 120.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:24 INFO DAGScheduler: ShuffleMapStage 120 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 120.371 s
25/02/06 06:11:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:24 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 142, ShuffleMapStage 244,
ShuffleMapStage 134, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 284, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 122, ShuffleMapStage 224,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:24 INFO DAGScheduler: failed: Set()
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 13 to 10.183.233.191:55538
25/02/06 06:11:24 INFO TaskSetManager: Finished task 5.0 in stage 122.0 (TID 586)
in 829 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 5.0 in stage 130.0 (TID 604)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:24 INFO ShufflePartitionsUtil: For shuffle(105), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:24 INFO ShufflePartitionsUtil: For shuffle(95), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:24 INFO ShufflePartitionsUtil: For shuffle(93), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:24 INFO ShufflePartitionsUtil: For shuffle(96), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:24 INFO TaskSetManager: Starting task 6.0 in stage 130.0 (TID 605)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 6.0 in stage 122.0 (TID 587)
in 902 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 13 to 10.183.246.72:39896
25/02/06 06:11:24 INFO ShufflePartitionsUtil: For shuffle(90, 8), advisory target
size: 67108864, actual target size 67108864, minimum partition size: 1048576
25/02/06 06:11:24 INFO TaskSetManager: Starting task 7.0 in stage 130.0 (TID 606)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 7.0 in stage 122.0 (TID 588)
in 878 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:24 INFO TaskSetManager: Finished task 2.0 in stage 122.0 (TID 583)
in 1035 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 8.0 in stage 130.0 (TID 607)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 0.0 in stage 122.0 (TID 581)
in 1040 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 0.0 in stage 128.0 (TID 608)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_18 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_18_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:24 INFO SparkContext: Created broadcast 18 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 21 to 10.183.229.123:55848
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 13 to 10.183.229.123:55848
25/02/06 06:11:24 INFO TaskSetManager: Starting task 1.0 in stage 128.0 (TID 609)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 3.0 in stage 122.0 (TID 584)
in 1100 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:24 INFO TaskSetManager: Finished task 1.0 in stage 126.0 (TID 591)
in 950 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 2.0 in stage 128.0 (TID 610)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 21 to 10.183.246.72:39896
25/02/06 06:11:24 INFO TaskSetManager: Starting task 3.0 in stage 128.0 (TID 611)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 4.0 in stage 126.0 (TID 594)
in 863 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 4.0 in stage 128.0 (TID 612)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 1.0 in stage 122.0 (TID 582)
in 1139 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:24 INFO TaskSchedulerImpl: Removed TaskSet 122.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:24 INFO DAGScheduler: ShuffleMapStage 122 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 120.684 s
25/02/06 06:11:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:24 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 144, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146,
ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 140, ShuffleMapStage 242,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 142, ShuffleMapStage 244,
ShuffleMapStage 134, ShuffleMapStage 236, ShuffleMapStage 186, ShuffleMapStage 136,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 138, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 284, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 126, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 174,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:24 INFO DAGScheduler: failed: Set()
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 21 to 10.183.233.191:55538
25/02/06 06:11:24 INFO TaskSetManager: Starting task 5.0 in stage 128.0 (TID 613)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 2.0 in stage 126.0 (TID 592)
in 1002 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:24 INFO TaskSetManager: Finished task 0.0 in stage 126.0 (TID 590)
in 1093 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 6.0 in stage 128.0 (TID 614)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Starting task 7.0 in stage 128.0 (TID 615)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 3.0 in stage 126.0 (TID 593)
in 1109 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 21 to 10.183.249.197:36834
25/02/06 06:11:24 INFO TaskSetManager: Finished task 7.0 in stage 126.0 (TID 597)
in 1063 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 8.0 in stage 128.0 (TID 616)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Starting task 0.0 in stage 132.0 (TID 617)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 8.0 in stage 130.0 (TID 607)
in 394 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_19 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO MemoryStore: Block broadcast_19_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:24 INFO SparkContext: Created broadcast 19 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 1 to 10.183.229.123:55848
25/02/06 06:11:24 INFO TaskSetManager: Finished task 5.0 in stage 126.0 (TID 595)
in 1181 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 1.0 in stage 132.0 (TID 618)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 1 to 10.183.249.197:36834
25/02/06 06:11:24 INFO TaskSetManager: Starting task 2.0 in stage 132.0 (TID 619)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:24 INFO TaskSetManager: Finished task 0.0 in stage 130.0 (TID 599)
in 851 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:24 INFO TaskSetManager: Starting task 3.0 in stage 132.0 (TID 620)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:24 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:24 INFO TaskSetManager: Finished task 5.0 in stage 130.0 (TID 604)
in 702 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 1 to 10.183.232.45:60256
25/02/06 06:11:25 INFO TaskSetManager: Starting task 4.0 in stage 132.0 (TID 621)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 1.0 in stage 130.0 (TID 600)
in 858 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:25 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2101162) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2101162)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:25 INFO CodeGenerator: Code generated in 33.923915 ms
25/02/06 06:11:25 INFO DAGScheduler: Registering RDD 1237 (mapPartitionsInternal at
PhotonExec.scala:483) as input to shuffle 190
25/02/06 06:11:25 INFO DAGScheduler: Got map stage job 193 (submitMapStage at
PhotonExec.scala:495) with 9 output partitions
25/02/06 06:11:25 INFO DAGScheduler: Final stage: ShuffleMapStage 302
(mapPartitionsInternal at PhotonExec.scala:483)
25/02/06 06:11:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
97, ShuffleMapStage 301)
25/02/06 06:11:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:25 INFO DAGScheduler: Submitting ShuffleMapStage 302
(MapPartitionsRDD[1237] at mapPartitionsInternal at PhotonExec.scala:483), which
has no missing parents
25/02/06 06:11:25 INFO DAGScheduler: Registering RDD 1239
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 191
25/02/06 06:11:25 INFO DAGScheduler: Got map stage job 194
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:25 INFO DAGScheduler: Final stage: ShuffleMapStage 305
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
304)
25/02/06 06:11:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:25 INFO DAGScheduler: Submitting ShuffleMapStage 305
(MapPartitionsRDD[1239] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:25 INFO DAGScheduler: Submitting 9 missing tasks from
ShuffleMapStage 302 (MapPartitionsRDD[1237] at mapPartitionsInternal at
PhotonExec.scala:483) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5,
6, 7, 8))
25/02/06 06:11:25 INFO TaskSchedulerImpl: Adding task set 302.0 with 9 tasks
resource profile 0
25/02/06 06:11:25 INFO TaskSetManager: TaskSet 302.0 using PreferredLocationsV1
25/02/06 06:11:25 INFO FairSchedulableBuilder: Added task set TaskSet_302.0 tasks
to pool 5400065285585087590
25/02/06 06:11:25 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 305 (MapPartitionsRDD[1239] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:25 INFO TaskSchedulerImpl: Adding task set 305.0 with 5 tasks
resource profile 0
25/02/06 06:11:25 INFO TaskSetManager: TaskSet 305.0 using PreferredLocationsV1
25/02/06 06:11:25 INFO FairSchedulableBuilder: Added task set TaskSet_305.0 tasks
to pool 5400065285585087590
25/02/06 06:11:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:25 INFO CodeGenerator: Code generated in 52.460252 ms
25/02/06 06:11:25 INFO CodeGenerator: Code generated in 72.815718 ms
25/02/06 06:11:25 INFO DAGScheduler: Registering RDD 1247
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 192
25/02/06 06:11:25 INFO DAGScheduler: Got map stage job 195
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:25 INFO DAGScheduler: Final stage: ShuffleMapStage 308
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
307)
25/02/06 06:11:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:25 INFO DAGScheduler: Submitting ShuffleMapStage 308
(MapPartitionsRDD[1247] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:25 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 308 (MapPartitionsRDD[1247] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:25 INFO TaskSchedulerImpl: Adding task set 308.0 with 6 tasks
resource profile 0
25/02/06 06:11:25 INFO TaskSetManager: TaskSet 308.0 using PreferredLocationsV1
25/02/06 06:11:25 INFO FairSchedulableBuilder: Added task set TaskSet_308.0 tasks
to pool 5400065285585087590
25/02/06 06:11:25 INFO DAGScheduler: Registering RDD 1249
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 193
25/02/06 06:11:25 INFO DAGScheduler: Got map stage job 196
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:25 INFO DAGScheduler: Final stage: ShuffleMapStage 311
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
310)
25/02/06 06:11:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:25 INFO DAGScheduler: Submitting ShuffleMapStage 311
(MapPartitionsRDD[1249] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:25 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 311 (MapPartitionsRDD[1249] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:25 INFO TaskSchedulerImpl: Adding task set 311.0 with 6 tasks
resource profile 0
25/02/06 06:11:25 INFO TaskSetManager: TaskSet 311.0 using PreferredLocationsV1
25/02/06 06:11:25 INFO FairSchedulableBuilder: Added task set TaskSet_311.0 tasks
to pool 5400065285585087590
25/02/06 06:11:25 INFO TaskSetManager: Starting task 5.0 in stage 132.0 (TID 622)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 7.0 in stage 130.0 (TID 606)
in 712 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:25 INFO CodeGenerator: Code generated in 95.625966 ms
25/02/06 06:11:25 INFO TaskSetManager: Finished task 6.0 in stage 126.0 (TID 596)
in 1401 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:25 INFO TaskSchedulerImpl: Removed TaskSet 126.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:25 INFO DAGScheduler: ShuffleMapStage 126 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 121.272 s
25/02/06 06:11:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:25 INFO TaskSetManager: Starting task 6.0 in stage 132.0 (TID 623)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:25 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 130, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 128, ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 162,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:25 INFO DAGScheduler: failed: Set()
25/02/06 06:11:25 INFO DAGScheduler: Registering RDD 1254
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 194
25/02/06 06:11:25 INFO DAGScheduler: Got map stage job 197
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:25 INFO DAGScheduler: Final stage: ShuffleMapStage 314
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
313)
25/02/06 06:11:25 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:25 INFO DAGScheduler: Submitting ShuffleMapStage 314
(MapPartitionsRDD[1254] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:25 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 314 (MapPartitionsRDD[1254] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:25 INFO TaskSchedulerImpl: Adding task set 314.0 with 6 tasks
resource profile 0
25/02/06 06:11:25 INFO TaskSetManager: TaskSet 314.0 using PreferredLocationsV1
25/02/06 06:11:25 INFO FairSchedulableBuilder: Added task set TaskSet_314.0 tasks
to pool 5400065285585087590
25/02/06 06:11:25 INFO TaskSetManager: Finished task 2.0 in stage 130.0 (TID 601)
in 985 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 7.0 in stage 132.0 (TID 624)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 8.0 in stage 132.0 (TID 625)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 8.0 in stage 128.0 (TID 616)
in 373 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 1 to 10.183.233.191:55538
25/02/06 06:11:25 INFO TaskSetManager: Starting task 0.0 in stage 134.0 (TID 626)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:25 INFO MemoryStore: Block broadcast_20 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:25 INFO MemoryStore: Block broadcast_20_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:25 INFO TaskSetManager: Finished task 3.0 in stage 130.0 (TID 602)
in 1054 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:25 INFO SparkContext: Created broadcast 20 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 18 to 10.183.232.45:60256
25/02/06 06:11:25 INFO TaskSetManager: Starting task 1.0 in stage 134.0 (TID 627)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 4.0 in stage 128.0 (TID 612)
in 768 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO TaskSetManager: Finished task 4.0 in stage 130.0 (TID 603)
in 1101 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 2.0 in stage 134.0 (TID 628)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 18 to 10.183.229.123:55848
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 3.0 in stage 134.0 (TID 629)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 6.0 in stage 130.0 (TID 605)
in 983 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:25 INFO TaskSchedulerImpl: Removed TaskSet 130.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:25 INFO DAGScheduler: ShuffleMapStage 130 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 121.444 s
25/02/06 06:11:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:25 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286,
ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 128,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:25 INFO DAGScheduler: failed: Set()
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 18 to 10.183.233.191:55538
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 18 to 10.183.246.72:39896
25/02/06 06:11:25 INFO TaskSetManager: Finished task 0.0 in stage 128.0 (TID 608)
in 961 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 4.0 in stage 134.0 (TID 630)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 2.0 in stage 128.0 (TID 610)
in 933 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 5.0 in stage 134.0 (TID 631)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Starting task 6.0 in stage 134.0 (TID 632)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 3.0 in stage 128.0 (TID 611)
in 989 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:25 INFO ShufflePartitionsUtil: For shuffle(94), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:25 INFO TaskSetManager: Starting task 7.0 in stage 134.0 (TID 633)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 6.0 in stage 128.0 (TID 614)
in 939 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:25 INFO ShufflePartitionsUtil: For shuffle(104), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:25 INFO TaskSetManager: Starting task 8.0 in stage 134.0 (TID 634)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 5.0 in stage 128.0 (TID 613)
in 995 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:25 INFO TaskSetManager: Starting task 0.0 in stage 136.0 (TID 635)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:25 INFO TaskSetManager: Finished task 8.0 in stage 132.0 (TID 625)
in 467 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:25 INFO MemoryStore: Block broadcast_21 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:25 INFO MemoryStore: Block broadcast_21_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:25 INFO SparkContext: Created broadcast 21 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:25 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 43 to 10.183.233.191:55538
25/02/06 06:11:26 INFO TaskSetManager: Finished task 2.0 in stage 132.0 (TID 619)
in 1177 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 1.0 in stage 136.0 (TID 636)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 2.0 in stage 136.0 (TID 637)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 3.0 in stage 136.0 (TID 638)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 4.0 in stage 136.0 (TID 639)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 5.0 in stage 136.0 (TID 640)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 6.0 in stage 136.0 (TID 641)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 7.0 in stage 136.0 (TID 642)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 5.0 in stage 132.0 (TID 622)
in 1006 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 0.0 in stage 134.0 (TID 626)
in 859 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 1.0 in stage 132.0 (TID 618)
in 1231 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 4.0 in stage 132.0 (TID 621)
in 1162 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 3.0 in stage 132.0 (TID 620)
in 1177 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 0.0 in stage 132.0 (TID 617)
in 1280 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 7.0 in stage 128.0 (TID 615)
in 1386 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 8.0 in stage 134.0 (TID 634)
in 512 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 8.0 in stage 136.0 (TID 643)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 0.0 in stage 138.0 (TID 644)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Starting task 1.0 in stage 138.0 (TID 645)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 1.0 in stage 128.0 (TID 609)
in 1618 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:26 INFO TaskSchedulerImpl: Removed TaskSet 128.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:26 INFO DAGScheduler: ShuffleMapStage 128 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 122.239 s
25/02/06 06:11:26 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:26 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 132, ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286,
ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:26 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:26 INFO DAGScheduler: failed: Set()
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MemoryStore: Block broadcast_22 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MemoryStore: Block broadcast_22_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 43 to 10.183.229.123:55848
25/02/06 06:11:26 INFO SparkContext: Created broadcast 22 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 43 to 10.183.232.45:60256
25/02/06 06:11:26 INFO TaskSetManager: Finished task 6.0 in stage 132.0 (TID 623)
in 1004 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 2.0 in stage 138.0 (TID 646)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 5 to 10.183.246.72:39896
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 5 to 10.183.229.123:55848
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 43 to 10.183.249.197:36834
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 5 to 10.183.249.197:36834
25/02/06 06:11:26 INFO TaskSetManager: Starting task 3.0 in stage 138.0 (TID 647)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 4.0 in stage 134.0 (TID 630)
in 801 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 4.0 in stage 138.0 (TID 648)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 1.0 in stage 134.0 (TID 627)
in 960 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 5.0 in stage 138.0 (TID 649)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 3.0 in stage 134.0 (TID 629)
in 938 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 6.0 in stage 138.0 (TID 650)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 7.0 in stage 132.0 (TID 624)
in 1166 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:26 INFO TaskSchedulerImpl: Removed TaskSet 132.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:26 INFO DAGScheduler: ShuffleMapStage 132 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 122.445 s
25/02/06 06:11:26 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:26 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 134, ShuffleMapStage 236,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 174,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:26 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:26 INFO DAGScheduler: failed: Set()
25/02/06 06:11:26 INFO TaskSetManager: Starting task 7.0 in stage 138.0 (TID 651)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 7.0 in stage 134.0 (TID 633)
in 773 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 5 to 10.183.233.191:55538
25/02/06 06:11:26 INFO TaskSetManager: Starting task 8.0 in stage 138.0 (TID 652)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 5.0 in stage 134.0 (TID 631)
in 928 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:26 INFO TaskSetManager: Finished task 2.0 in stage 134.0 (TID 628)
in 1075 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:26 INFO TaskSetManager: Starting task 0.0 in stage 140.0 (TID 653)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:26 INFO MemoryStore: Block broadcast_23 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:26 INFO MemoryStore: Block broadcast_23_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:26 INFO SparkContext: Created broadcast 23 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 3 to 10.183.233.191:55538
25/02/06 06:11:26 INFO TaskSetManager: Starting task 1.0 in stage 140.0 (TID 654)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 8.0 in stage 136.0 (TID 643)
in 407 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 3 to 10.183.229.123:55848
25/02/06 06:11:26 INFO TaskSetManager: Starting task 2.0 in stage 140.0 (TID 655)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 6.0 in stage 134.0 (TID 632)
in 1092 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:26 INFO TaskSchedulerImpl: Removed TaskSet 134.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:26 INFO DAGScheduler: ShuffleMapStage 134 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 122.673 s
25/02/06 06:11:26 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:26 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 186,
ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 138, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:26 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:26 INFO DAGScheduler: failed: Set()
25/02/06 06:11:26 INFO TaskSetManager: Starting task 3.0 in stage 140.0 (TID 656)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 0.0 in stage 136.0 (TID 635)
in 958 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:26 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098302) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098302)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:26 INFO CodeGenerator: Code generated in 40.394364 ms
25/02/06 06:11:26 INFO CodeGenerator: Code generated in 56.642854 ms
25/02/06 06:11:26 INFO TaskSetManager: Starting task 4.0 in stage 140.0 (TID 657)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:26 INFO TaskSetManager: Finished task 8.0 in stage 138.0 (TID 652)
in 359 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:26 INFO DAGScheduler: Registering RDD 1262
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 195
25/02/06 06:11:26 INFO DAGScheduler: Got map stage job 198
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:26 INFO DAGScheduler: Final stage: ShuffleMapStage 317
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
316)
25/02/06 06:11:26 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:26 INFO DAGScheduler: Submitting ShuffleMapStage 317
(MapPartitionsRDD[1262] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:26 INFO DAGScheduler: Registering RDD 1264
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 196
25/02/06 06:11:26 INFO DAGScheduler: Got map stage job 199
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:26 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:26 INFO DAGScheduler: Final stage: ShuffleMapStage 320
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
319)
25/02/06 06:11:26 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:26 INFO DAGScheduler: Submitting ShuffleMapStage 320
(MapPartitionsRDD[1264] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:26 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 317 (MapPartitionsRDD[1262] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:26 INFO TaskSchedulerImpl: Adding task set 317.0 with 6 tasks
resource profile 0
25/02/06 06:11:26 INFO TaskSetManager: TaskSet 317.0 using PreferredLocationsV1
25/02/06 06:11:26 INFO FairSchedulableBuilder: Added task set TaskSet_317.0 tasks
to pool 5400065285585087590
25/02/06 06:11:26 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 320 (MapPartitionsRDD[1264] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:26 INFO TaskSchedulerImpl: Adding task set 320.0 with 6 tasks
resource profile 0
25/02/06 06:11:26 INFO TaskSetManager: TaskSet 320.0 using PreferredLocationsV1
25/02/06 06:11:26 INFO FairSchedulableBuilder: Added task set TaskSet_320.0 tasks
to pool 5400065285585087590
25/02/06 06:11:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 3 to 10.183.246.72:39896
25/02/06 06:11:27 INFO TaskSetManager: Starting task 5.0 in stage 140.0 (TID 658)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 1.0 in stage 138.0 (TID 645)
in 861 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 6.0 in stage 140.0 (TID 659)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 3.0 in stage 138.0 (TID 647)
in 795 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 4.0 in stage 136.0 (TID 639)
in 914 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 7.0 in stage 140.0 (TID 660)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 3 to 10.183.232.45:60256
25/02/06 06:11:27 INFO ShufflePartitionsUtil: For shuffle(97), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:27 INFO ShufflePartitionsUtil: For shuffle(100), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:27 INFO TaskSetManager: Starting task 8.0 in stage 140.0 (TID 661)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 2.0 in stage 136.0 (TID 637)
in 958 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:27 INFO ShufflePartitionsUtil: For shuffle(111), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:27 INFO ShufflePartitionsUtil: For shuffle(109), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:27 INFO ShufflePartitionsUtil: For shuffle(110), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:27 INFO TaskSetManager: Starting task 0.0 in stage 144.0 (TID 662)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 5.0 in stage 138.0 (TID 649)
in 864 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:27 INFO MemoryStore: Block broadcast_24 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:27 INFO MemoryStore: Block broadcast_24_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:27 INFO SparkContext: Created broadcast 24 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:27 INFO TaskSetManager: Finished task 1.0 in stage 136.0 (TID 636)
in 1045 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 1.0 in stage 144.0 (TID 663)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Starting task 2.0 in stage 144.0 (TID 664)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 3.0 in stage 136.0 (TID 638)
in 1050 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 12 to 10.183.232.45:60256
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 12 to 10.183.246.72:39896
25/02/06 06:11:27 INFO TaskSetManager: Starting task 3.0 in stage 144.0 (TID 665)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 7.0 in stage 138.0 (TID 651)
in 844 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 4.0 in stage 138.0 (TID 648)
in 969 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 4.0 in stage 144.0 (TID 666)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 5.0 in stage 144.0 (TID 667)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 0.0 in stage 140.0 (TID 653)
in 848 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 12 to 10.183.229.123:55848
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 12 to 10.183.233.191:55538
25/02/06 06:11:27 INFO TaskSetManager: Starting task 6.0 in stage 144.0 (TID 668)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 1.0 in stage 140.0 (TID 654)
in 760 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 7.0 in stage 144.0 (TID 669)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 0.0 in stage 138.0 (TID 644)
in 1181 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 5.0 in stage 136.0 (TID 640)
in 1227 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 8.0 in stage 144.0 (TID 670)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Starting task 0.0 in stage 142.0 (TID 671)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Starting task 1.0 in stage 142.0 (TID 672)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 6.0 in stage 136.0 (TID 641)
in 1248 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 2.0 in stage 138.0 (TID 646)
in 1204 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:27 INFO MemoryStore: Block broadcast_25 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 2.0 in stage 140.0 (TID 655)
in 743 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 2.0 in stage 142.0 (TID 673)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:27 INFO MemoryStore: Block broadcast_25_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:27 INFO SparkContext: Created broadcast 25 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 3.0 in stage 142.0 (TID 674)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 6.0 in stage 138.0 (TID 650)
in 1023 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:27 INFO TaskSchedulerImpl: Removed TaskSet 138.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:27 INFO DAGScheduler: ShuffleMapStage 138 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 123.394 s
25/02/06 06:11:27 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:27 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 136, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:27 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:27 INFO DAGScheduler: failed: Set()
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 9 to 10.183.233.191:55538
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 12 to 10.183.249.197:36834
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 9 to 10.183.249.197:36834
25/02/06 06:11:27 INFO TaskSetManager: Finished task 7.0 in stage 136.0 (TID 642)
in 1400 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:27 INFO TaskSchedulerImpl: Removed TaskSet 136.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:27 INFO DAGScheduler: ShuffleMapStage 136 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 123.544 s
25/02/06 06:11:27 INFO TaskSetManager: Starting task 4.0 in stage 142.0 (TID 675)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:27 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:27 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 140, ShuffleMapStage 242, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 224, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:27 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:27 INFO DAGScheduler: failed: Set()
25/02/06 06:11:27 INFO TaskSetManager: Starting task 5.0 in stage 142.0 (TID 676)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 8.0 in stage 140.0 (TID 661)
in 474 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 9 to 10.183.232.45:60256
25/02/06 06:11:27 INFO TaskSetManager: Starting task 6.0 in stage 142.0 (TID 677)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:27 INFO TaskSetManager: Finished task 3.0 in stage 140.0 (TID 656)
in 1032 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:27 INFO TaskSetManager: Finished task 4.0 in stage 140.0 (TID 657)
in 919 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 7.0 in stage 142.0 (TID 678)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:27 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 9 to 10.183.246.72:39896
25/02/06 06:11:27 INFO TaskSetManager: Finished task 7.0 in stage 140.0 (TID 660)
in 740 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:27 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099183) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099183)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:27 INFO TaskSetManager: Starting task 8.0 in stage 142.0 (TID 679)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:27 INFO CodeGenerator: Code generated in 48.196974 ms
25/02/06 06:11:27 INFO CodeGenerator: Code generated in 65.282356 ms
25/02/06 06:11:27 INFO CodeGenerator: Code generated in 65.743496 ms
25/02/06 06:11:27 INFO CodeGenerator: Code generated in 57.915125 ms
25/02/06 06:11:27 INFO DAGScheduler: Registering RDD 1272
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 197
25/02/06 06:11:27 INFO DAGScheduler: Got map stage job 200
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:27 INFO DAGScheduler: Final stage: ShuffleMapStage 323
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:27 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
322)
25/02/06 06:11:27 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:27 INFO DAGScheduler: Submitting ShuffleMapStage 323
(MapPartitionsRDD[1272] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:27 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 323 (MapPartitionsRDD[1272] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:27 INFO TaskSchedulerImpl: Adding task set 323.0 with 5 tasks
resource profile 0
25/02/06 06:11:27 INFO TaskSetManager: TaskSet 323.0 using PreferredLocationsV1
25/02/06 06:11:27 INFO FairSchedulableBuilder: Added task set TaskSet_323.0 tasks
to pool 5400065285585087590
25/02/06 06:11:27 INFO DAGScheduler: Registering RDD 1274
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 198
25/02/06 06:11:27 INFO DAGScheduler: Got map stage job 201
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:27 INFO DAGScheduler: Final stage: ShuffleMapStage 326
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:27 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
325)
25/02/06 06:11:27 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:27 INFO DAGScheduler: Submitting ShuffleMapStage 326
(MapPartitionsRDD[1274] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:27 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 326 (MapPartitionsRDD[1274] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:27 INFO TaskSchedulerImpl: Adding task set 326.0 with 6 tasks
resource profile 0
25/02/06 06:11:27 INFO TaskSetManager: TaskSet 326.0 using PreferredLocationsV1
25/02/06 06:11:27 INFO FairSchedulableBuilder: Added task set TaskSet_326.0 tasks
to pool 5400065285585087590
25/02/06 06:11:27 INFO TaskSetManager: Finished task 6.0 in stage 140.0 (TID 659)
in 839 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:27 INFO TaskSetManager: Starting task 0.0 in stage 146.0 (TID 680)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:27 INFO MemoryStore: Block broadcast_26 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:28 INFO TaskSetManager: Starting task 1.0 in stage 146.0 (TID 681)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Finished task 5.0 in stage 140.0 (TID 658)
in 1211 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:28 INFO TaskSchedulerImpl: Removed TaskSet 140.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:28 INFO TaskSetManager: Finished task 0.0 in stage 144.0 (TID 662)
in 1054 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:28 INFO DAGScheduler: ShuffleMapStage 140 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 124.213 s
25/02/06 06:11:28 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:28 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 148, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 144, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 230, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 224, ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:28 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:28 INFO DAGScheduler: failed: Set()
25/02/06 06:11:28 INFO DAGScheduler: Registering RDD 1279
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 199
25/02/06 06:11:28 INFO DAGScheduler: Got map stage job 202
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:28 INFO DAGScheduler: Final stage: ShuffleMapStage 329
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:28 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
328)
25/02/06 06:11:28 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:28 INFO DAGScheduler: Submitting ShuffleMapStage 329
(MapPartitionsRDD[1279] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:28 INFO CodeGenerator: Code generated in 418.311466 ms
25/02/06 06:11:28 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 329 (MapPartitionsRDD[1279] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:28 INFO TaskSchedulerImpl: Adding task set 329.0 with 6 tasks
resource profile 0
25/02/06 06:11:28 INFO TaskSetManager: Finished task 0.0 in stage 142.0 (TID 671)
in 861 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 4.0 in stage 144.0 (TID 666)
in 978 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:28 INFO MemoryStore: Block broadcast_26_piece0 stored as bytes in
memory (estimated size 38.6 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO TaskSetManager: TaskSet 329.0 using PreferredLocationsV1
25/02/06 06:11:28 INFO FairSchedulableBuilder: Added task set TaskSet_329.0 tasks
to pool 5400065285585087590
25/02/06 06:11:28 INFO TaskSetManager: Starting task 2.0 in stage 146.0 (TID 682)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 3.0 in stage 146.0 (TID 683)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 4.0 in stage 146.0 (TID 684)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 5.0 in stage 146.0 (TID 685)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 6.0 in stage 146.0 (TID 686)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on
10.183.232.44:37705 (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 8.0 in stage 144.0 (TID 670)
in 891 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 6.0 in stage 144.0 (TID 668)
in 941 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 8.0 in stage 142.0 (TID 679)
in 462 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 2.0 in stage 144.0 (TID 664)
in 1071 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 1.0 in stage 144.0 (TID 663)
in 1074 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 5.0 in stage 142.0 (TID 676)
in 690 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 3.0 in stage 144.0 (TID 665)
in 1031 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 1.0 in stage 142.0 (TID 672)
in 875 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:28 INFO SparkContext: Created broadcast 26 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:28 INFO TaskSetManager: Starting task 7.0 in stage 146.0 (TID 687)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 8.0 in stage 146.0 (TID 688)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 0.0 in stage 148.0 (TID 689)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 1.0 in stage 148.0 (TID 690)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:28 INFO MemoryStore: Block broadcast_27 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO DAGScheduler: Registering RDD 1284
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 200
25/02/06 06:11:28 INFO DAGScheduler: Got map stage job 203
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:28 INFO DAGScheduler: Final stage: ShuffleMapStage 332
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:28 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
331)
25/02/06 06:11:28 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:28 INFO TaskSetManager: Starting task 2.0 in stage 148.0 (TID 691)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Starting task 3.0 in stage 148.0 (TID 692)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:28 INFO DAGScheduler: Submitting ShuffleMapStage 332
(MapPartitionsRDD[1284] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on
10.183.246.72:43231 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 332 (MapPartitionsRDD[1284] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:28 INFO TaskSchedulerImpl: Adding task set 332.0 with 5 tasks
resource profile 0
25/02/06 06:11:28 INFO TaskSetManager: TaskSet 332.0 using PreferredLocationsV1
25/02/06 06:11:28 INFO FairSchedulableBuilder: Added task set TaskSet_332.0 tasks
to pool 5400065285585087590
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on
10.183.249.197:40023 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO MemoryStore: Block broadcast_27_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:28 INFO SparkContext: Created broadcast 27 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:28 INFO TaskSetManager: Finished task 7.0 in stage 144.0 (TID 669)
in 951 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on
10.183.229.123:38441 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 7.0 in stage 142.0 (TID 678)
in 599 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on
10.183.232.45:45579 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO DAGScheduler: Registering RDD 1289
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 201
25/02/06 06:11:28 INFO DAGScheduler: Got map stage job 204
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:28 INFO TaskSetManager: Starting task 4.0 in stage 148.0 (TID 693)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:28 INFO DAGScheduler: Final stage: ShuffleMapStage 335
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:28 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
334)
25/02/06 06:11:28 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:28 INFO TaskSetManager: Starting task 5.0 in stage 148.0 (TID 694)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:28 INFO DAGScheduler: Submitting ShuffleMapStage 335
(MapPartitionsRDD[1289] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 2 to 10.183.229.123:55848
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 2 to 10.183.246.72:39896
25/02/06 06:11:28 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 335 (MapPartitionsRDD[1289] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:28 INFO TaskSchedulerImpl: Adding task set 335.0 with 6 tasks
resource profile 0
25/02/06 06:11:28 INFO TaskSetManager: TaskSet 335.0 using PreferredLocationsV1
25/02/06 06:11:28 INFO FairSchedulableBuilder: Added task set TaskSet_335.0 tasks
to pool 5400065285585087590
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 2 to 10.183.249.197:36834
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 2 to 10.183.232.45:60256
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 4 to 10.183.232.45:60256
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO TaskSetManager: Starting task 6.0 in stage 148.0 (TID 695)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Finished task 5.0 in stage 144.0 (TID 667)
in 1028 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:28 INFO TaskSchedulerImpl: Removed TaskSet 144.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:28 INFO DAGScheduler: ShuffleMapStage 144 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 124.248 s
25/02/06 06:11:28 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:28 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 250,
ShuffleMapStage 200, ShuffleMapStage 302, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 323, ShuffleMapStage 192,
ShuffleMapStage 294, ShuffleMapStage 142, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:28 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:28 INFO DAGScheduler: failed: Set()
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 4 to 10.183.246.72:39896
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 4 to 10.183.229.123:55848
25/02/06 06:11:28 INFO TaskSetManager: Starting task 7.0 in stage 148.0 (TID 696)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Finished task 3.0 in stage 142.0 (TID 674)
in 923 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 4 to 10.183.233.191:55538
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 4 to 10.183.249.197:36834
25/02/06 06:11:28 INFO TaskSetManager: Starting task 8.0 in stage 148.0 (TID 697)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:28 INFO TaskSetManager: Finished task 4.0 in stage 142.0 (TID 675)
in 913 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:28 INFO TaskSetManager: Starting task 0.0 in stage 150.0 (TID 698)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:28 INFO MemoryStore: Block broadcast_28 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:28 INFO TaskSetManager: Finished task 2.0 in stage 142.0 (TID 673)
in 1134 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:28 INFO SparkContext: Created broadcast 28 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 41 to 10.183.233.191:55538
25/02/06 06:11:28 INFO TaskSetManager: Finished task 8.0 in stage 146.0 (TID 688)
in 350 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:28 INFO TaskSetManager: Starting task 1.0 in stage 150.0 (TID 699)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 41 to 10.183.232.45:60256
25/02/06 06:11:28 INFO ShufflePartitionsUtil: For shuffle(113), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:28 INFO ShufflePartitionsUtil: For shuffle(112), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:28 INFO TaskSetManager: Finished task 6.0 in stage 142.0 (TID 677)
in 1020 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:28 INFO TaskSchedulerImpl: Removed TaskSet 142.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:28 INFO DAGScheduler: ShuffleMapStage 142 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 124.662 s
25/02/06 06:11:28 INFO TaskSetManager: Starting task 2.0 in stage 150.0 (TID 700)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:28 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:28 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 148, ShuffleMapStage 250,
ShuffleMapStage 200, ShuffleMapStage 302, ShuffleMapStage 150, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 323, ShuffleMapStage 192,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 326, ShuffleMapStage 174,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:28 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:28 INFO DAGScheduler: failed: Set()
25/02/06 06:11:28 INFO TaskSetManager: Finished task 8.0 in stage 148.0 (TID 697)
in 454 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:28 INFO TaskSetManager: Starting task 3.0 in stage 150.0 (TID 701)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:28 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 41 to 10.183.249.197:36834
25/02/06 06:11:29 INFO TaskSetManager: Starting task 4.0 in stage 150.0 (TID 702)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 1.0 in stage 146.0 (TID 681)
in 851 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 5.0 in stage 150.0 (TID 703)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 0.0 in stage 148.0 (TID 689)
in 839 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:29 INFO TaskSetManager: Finished task 2.0 in stage 146.0 (TID 682)
in 854 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 6.0 in stage 150.0 (TID 704)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 41 to 10.183.246.72:39896
25/02/06 06:11:29 INFO TaskSetManager: Starting task 7.0 in stage 150.0 (TID 705)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 6.0 in stage 148.0 (TID 695)
in 824 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 8.0 in stage 150.0 (TID 706)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 7.0 in stage 148.0 (TID 696)
in 891 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:29 INFO TaskSetManager: Finished task 1.0 in stage 148.0 (TID 690)
in 957 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 0.0 in stage 152.0 (TID 707)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:29 INFO MemoryStore: Block broadcast_29 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:29 INFO MemoryStore: Block broadcast_29_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:29 INFO SparkContext: Created broadcast 29 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:29 INFO TaskSetManager: Starting task 1.0 in stage 152.0 (TID 708)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 5.0 in stage 148.0 (TID 694)
in 939 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 37 to 10.183.232.45:60256
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 37 to 10.183.246.72:39896
25/02/06 06:11:29 INFO TaskSetManager: Finished task 5.0 in stage 146.0 (TID 685)
in 1009 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 2.0 in stage 152.0 (TID 709)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 37 to 10.183.249.197:36834
25/02/06 06:11:29 INFO TaskSetManager: Starting task 3.0 in stage 152.0 (TID 710)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 3.0 in stage 148.0 (TID 692)
in 1128 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 4.0 in stage 152.0 (TID 711)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 2.0 in stage 148.0 (TID 691)
in 1148 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:29 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2101258) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2101258)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:29 INFO TaskSetManager: Starting task 5.0 in stage 152.0 (TID 712)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 7.0 in stage 146.0 (TID 687)
in 1181 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:29 INFO CodeGenerator: Code generated in 62.188271 ms
25/02/06 06:11:29 INFO TaskSetManager: Starting task 6.0 in stage 152.0 (TID 713)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 0.0 in stage 150.0 (TID 698)
in 932 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 7.0 in stage 152.0 (TID 714)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 6.0 in stage 146.0 (TID 686)
in 1200 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:29 INFO DAGScheduler: Registering RDD 1294
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 202
25/02/06 06:11:29 INFO DAGScheduler: Got map stage job 205
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:29 INFO DAGScheduler: Final stage: ShuffleMapStage 338
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
337)
25/02/06 06:11:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:29 INFO DAGScheduler: Submitting ShuffleMapStage 338
(MapPartitionsRDD[1294] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 338 (MapPartitionsRDD[1294] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:29 INFO TaskSchedulerImpl: Adding task set 338.0 with 6 tasks
resource profile 0
25/02/06 06:11:29 INFO TaskSetManager: TaskSet 338.0 using PreferredLocationsV1
25/02/06 06:11:29 INFO FairSchedulableBuilder: Added task set TaskSet_338.0 tasks
to pool 5400065285585087590
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 37 to 10.183.233.191:55538
25/02/06 06:11:29 INFO CodeGenerator: Code generated in 99.478512 ms
25/02/06 06:11:29 INFO DAGScheduler: Registering RDD 1299
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 203
25/02/06 06:11:29 INFO DAGScheduler: Got map stage job 206
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:29 INFO DAGScheduler: Final stage: ShuffleMapStage 341
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
340)
25/02/06 06:11:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:29 INFO DAGScheduler: Submitting ShuffleMapStage 341
(MapPartitionsRDD[1299] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 341 (MapPartitionsRDD[1299] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:29 INFO TaskSchedulerImpl: Adding task set 341.0 with 5 tasks
resource profile 0
25/02/06 06:11:29 INFO TaskSetManager: TaskSet 341.0 using PreferredLocationsV1
25/02/06 06:11:29 INFO FairSchedulableBuilder: Added task set TaskSet_341.0 tasks
to pool 5400065285585087590
25/02/06 06:11:29 INFO TaskSetManager: Finished task 4.0 in stage 148.0 (TID 693)
in 1222 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:29 INFO TaskSchedulerImpl: Removed TaskSet 148.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:29 INFO TaskSetManager: Starting task 8.0 in stage 152.0 (TID 715)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:29 INFO DAGScheduler: ShuffleMapStage 148 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 125.410 s
25/02/06 06:11:29 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:29 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 146, ShuffleMapStage 248, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 326, ShuffleMapStage 174,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158,
ShuffleMapStage 260, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 160,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:29 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:29 INFO DAGScheduler: failed: Set()
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 37 to 10.183.229.123:55848
25/02/06 06:11:29 INFO TaskSetManager: Starting task 0.0 in stage 154.0 (TID 716)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 0.0 in stage 146.0 (TID 680)
in 1705 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:29 INFO MemoryStore: Block broadcast_30 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:29 INFO MemoryStore: Block broadcast_30_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:29 INFO SparkContext: Created broadcast 30 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 1.0 in stage 154.0 (TID 717)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 8.0 in stage 150.0 (TID 706)
in 374 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 47 to 10.183.229.123:55848
25/02/06 06:11:29 INFO TaskSetManager: Finished task 4.0 in stage 146.0 (TID 684)
in 1348 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 2.0 in stage 154.0 (TID 718)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 3.0 in stage 154.0 (TID 719)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 1.0 in stage 150.0 (TID 699)
in 1001 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 47 to 10.183.233.191:55538
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 47 to 10.183.232.45:60256
25/02/06 06:11:29 INFO TaskSetManager: Starting task 4.0 in stage 154.0 (TID 720)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 3.0 in stage 146.0 (TID 683)
in 1417 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:29 INFO TaskSchedulerImpl: Removed TaskSet 146.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:29 INFO DAGScheduler: ShuffleMapStage 146 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 125.582 s
25/02/06 06:11:29 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:29 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 150, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 300,
ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186,
ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286,
ShuffleMapStage 176, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172,
ShuffleMapStage 224, ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:29 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:29 INFO DAGScheduler: failed: Set()
25/02/06 06:11:29 INFO ShufflePartitionsUtil: For shuffle(116), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:29 INFO ShufflePartitionsUtil: For shuffle(114), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:29 INFO ShufflePartitionsUtil: For shuffle(115), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:29 INFO TaskSetManager: Starting task 5.0 in stage 154.0 (TID 721)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 2.0 in stage 150.0 (TID 700)
in 1107 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 6.0 in stage 154.0 (TID 722)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 3.0 in stage 150.0 (TID 701)
in 927 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 47 to 10.183.249.197:36834
25/02/06 06:11:29 INFO TaskSetManager: Finished task 8.0 in stage 152.0 (TID 715)
in 397 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:29 INFO TaskSetManager: Starting task 7.0 in stage 154.0 (TID 723)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Starting task 8.0 in stage 154.0 (TID 724)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:29 INFO TaskSetManager: Finished task 6.0 in stage 150.0 (TID 704)
in 822 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:29 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:29 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 47 to 10.183.246.72:39896
25/02/06 06:11:30 INFO TaskSetManager: Starting task 0.0 in stage 156.0 (TID 725)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 1.0 in stage 152.0 (TID 708)
in 810 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_31 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_31_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:30 INFO SparkContext: Created broadcast 31 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 42 to 10.183.246.72:39896
25/02/06 06:11:30 INFO TaskSetManager: Starting task 1.0 in stage 156.0 (TID 726)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 7.0 in stage 150.0 (TID 705)
in 1013 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 42 to 10.183.233.191:55538
25/02/06 06:11:30 INFO TaskSetManager: Starting task 2.0 in stage 156.0 (TID 727)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 3.0 in stage 156.0 (TID 728)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 4.0 in stage 156.0 (TID 729)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 5.0 in stage 156.0 (TID 730)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 6.0 in stage 156.0 (TID 731)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 7.0 in stage 156.0 (TID 732)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 5.0 in stage 152.0 (TID 712)
in 998 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 5.0 in stage 150.0 (TID 703)
in 1341 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 3.0 in stage 152.0 (TID 710)
in 1049 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 2.0 in stage 152.0 (TID 709)
in 1181 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 7.0 in stage 152.0 (TID 714)
in 991 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 4.0 in stage 152.0 (TID 711)
in 1031 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 8.0 in stage 154.0 (TID 724)
in 519 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 4.0 in stage 150.0 (TID 702)
in 1372 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:30 INFO TaskSchedulerImpl: Removed TaskSet 150.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:30 INFO TaskSetManager: Finished task 0.0 in stage 152.0 (TID 707)
in 1226 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:30 INFO DAGScheduler: ShuffleMapStage 150 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 126.333 s
25/02/06 06:11:30 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:30 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 152, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 248, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224,
ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 341,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:30 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:30 INFO DAGScheduler: failed: Set()
25/02/06 06:11:30 INFO TaskSetManager: Finished task 6.0 in stage 152.0 (TID 713)
in 1001 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:30 INFO TaskSchedulerImpl: Removed TaskSet 152.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:30 INFO TaskSetManager: Finished task 0.0 in stage 154.0 (TID 716)
in 875 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 2.0 in stage 154.0 (TID 718)
in 850 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:30 INFO DAGScheduler: ShuffleMapStage 152 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 126.314 s
25/02/06 06:11:30 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:30 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 154, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 323, ShuffleMapStage 192,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:30 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:30 INFO DAGScheduler: failed: Set()
25/02/06 06:11:30 INFO TaskSetManager: Starting task 8.0 in stage 156.0 (TID 733)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 0.0 in stage 158.0 (TID 734)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 1.0 in stage 158.0 (TID 735)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 2.0 in stage 158.0 (TID 736)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 3.0 in stage 158.0 (TID 737)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Starting task 4.0 in stage 158.0 (TID 738)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_32 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 3.0 in stage 154.0 (TID 719)
in 844 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO TaskSetManager: Starting task 5.0 in stage 158.0 (TID 739)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 42 to 10.183.232.45:60256
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_32_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:30 INFO SparkContext: Created broadcast 32 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 38 to 10.183.233.191:55538
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 42 to 10.183.249.197:36834
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 38 to 10.183.229.123:55848
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 38 to 10.183.246.72:39896
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 38 to 10.183.232.45:60256
25/02/06 06:11:30 INFO TaskSetManager: Starting task 6.0 in stage 158.0 (TID 740)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 1.0 in stage 154.0 (TID 717)
in 1008 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:30 INFO TaskSetManager: Starting task 7.0 in stage 158.0 (TID 741)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 4.0 in stage 154.0 (TID 720)
in 978 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 6.0 in stage 154.0 (TID 722)
in 862 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:30 INFO TaskSetManager: Starting task 8.0 in stage 158.0 (TID 742)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:30 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098015) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098015)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO TaskSetManager: Finished task 5.0 in stage 154.0 (TID 721)
in 901 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:30 INFO TaskSetManager: Starting task 0.0 in stage 160.0 (TID 743)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_33 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:30 INFO SparkContext: Created broadcast 33 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 38 to 10.183.249.197:36834
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 25 to 10.183.233.191:55538
25/02/06 06:11:30 INFO TaskSetManager: Starting task 1.0 in stage 160.0 (TID 744)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:30 INFO TaskSetManager: Finished task 8.0 in stage 156.0 (TID 733)
in 287 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:30 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:30 INFO CodeGenerator: Code generated in 48.803652 ms
25/02/06 06:11:30 INFO CodeGenerator: Code generated in 67.381447 ms
25/02/06 06:11:30 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 25 to 10.183.246.72:39896
25/02/06 06:11:30 INFO CodeGenerator: Code generated in 57.169717 ms
25/02/06 06:11:30 INFO DAGScheduler: Registering RDD 1312
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 205
25/02/06 06:11:30 INFO DAGScheduler: Got map stage job 207
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:30 INFO DAGScheduler: Final stage: ShuffleMapStage 344
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:30 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
343)
25/02/06 06:11:30 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:30 INFO DAGScheduler: Submitting ShuffleMapStage 344
(MapPartitionsRDD[1312] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:30 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 344 (MapPartitionsRDD[1312] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:30 INFO TaskSchedulerImpl: Adding task set 344.0 with 6 tasks
resource profile 0
25/02/06 06:11:30 INFO TaskSetManager: TaskSet 344.0 using PreferredLocationsV1
25/02/06 06:11:30 INFO FairSchedulableBuilder: Added task set TaskSet_344.0 tasks
to pool 5400065285585087590
25/02/06 06:11:30 INFO DAGScheduler: Registering RDD 1314
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 206
25/02/06 06:11:30 INFO DAGScheduler: Got map stage job 208
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:30 INFO DAGScheduler: Final stage: ShuffleMapStage 347
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:30 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
346)
25/02/06 06:11:30 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:30 INFO DAGScheduler: Submitting ShuffleMapStage 347
(MapPartitionsRDD[1314] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:30 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 347 (MapPartitionsRDD[1314] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:30 INFO TaskSchedulerImpl: Adding task set 347.0 with 6 tasks
resource profile 0
25/02/06 06:11:30 INFO TaskSetManager: TaskSet 347.0 using PreferredLocationsV1
25/02/06 06:11:30 INFO FairSchedulableBuilder: Added task set TaskSet_347.0 tasks
to pool 5400065285585087590
25/02/06 06:11:30 INFO DAGScheduler: Registering RDD 1309
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 204
25/02/06 06:11:30 INFO DAGScheduler: Got map stage job 209
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:30 INFO DAGScheduler: Final stage: ShuffleMapStage 350
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:30 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
349)
25/02/06 06:11:30 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:30 INFO DAGScheduler: Submitting ShuffleMapStage 350
(MapPartitionsRDD[1309] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:30 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 350 (MapPartitionsRDD[1309] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:30 INFO TaskSchedulerImpl: Adding task set 350.0 with 5 tasks
resource profile 0
25/02/06 06:11:30 INFO TaskSetManager: TaskSet 350.0 using PreferredLocationsV1
25/02/06 06:11:30 INFO FairSchedulableBuilder: Added task set TaskSet_350.0 tasks
to pool 5400065285585087590
25/02/06 06:11:30 INFO TaskSetManager: Finished task 0.0 in stage 156.0 (TID 725)
in 819 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:30 INFO TaskSetManager: Starting task 2.0 in stage 160.0 (TID 745)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Starting task 3.0 in stage 160.0 (TID 746)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 7.0 in stage 154.0 (TID 723)
in 1151 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:31 INFO TaskSchedulerImpl: Removed TaskSet 154.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:31 INFO DAGScheduler: ShuffleMapStage 154 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 126.922 s
25/02/06 06:11:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:31 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 156, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170,
ShuffleMapStage 314, ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:31 INFO DAGScheduler: failed: Set()
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 25 to 10.183.229.123:55848
25/02/06 06:11:31 INFO TaskSetManager: Starting task 4.0 in stage 160.0 (TID 747)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 1.0 in stage 156.0 (TID 726)
in 936 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:31 INFO TaskSetManager: Starting task 5.0 in stage 160.0 (TID 748)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 8.0 in stage 158.0 (TID 742)
in 400 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 25 to 10.183.249.197:36834
25/02/06 06:11:31 INFO ShufflePartitionsUtil: For shuffle(117), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:31 INFO ShufflePartitionsUtil: For shuffle(118), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:31 INFO ShufflePartitionsUtil: For shuffle(119), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:31 INFO ShufflePartitionsUtil: For shuffle(120), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:31 INFO TaskSetManager: Finished task 2.0 in stage 156.0 (TID 727)
in 773 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 6.0 in stage 160.0 (TID 749)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Starting task 7.0 in stage 160.0 (TID 750)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 0.0 in stage 158.0 (TID 734)
in 762 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 25 to 10.183.232.45:60256
25/02/06 06:11:31 INFO TaskSetManager: Starting task 8.0 in stage 160.0 (TID 751)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 1.0 in stage 158.0 (TID 735)
in 778 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 0.0 in stage 162.0 (TID 752)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 2.0 in stage 158.0 (TID 736)
in 811 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:31 INFO MemoryStore: Block broadcast_34 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:31 INFO MemoryStore: Block broadcast_34_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:31 INFO SparkContext: Created broadcast 34 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 1.0 in stage 162.0 (TID 753)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 7.0 in stage 156.0 (TID 732)
in 853 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 26 to 10.183.233.191:55538
25/02/06 06:11:31 INFO TaskSetManager: Starting task 2.0 in stage 162.0 (TID 754)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 4.0 in stage 156.0 (TID 729)
in 863 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 26 to 10.183.249.197:36834
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 26 to 10.183.232.45:60256
25/02/06 06:11:31 INFO TaskSetManager: Starting task 3.0 in stage 162.0 (TID 755)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 6.0 in stage 158.0 (TID 740)
in 764 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 4.0 in stage 162.0 (TID 756)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 3.0 in stage 158.0 (TID 737)
in 908 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO TaskSetManager: Finished task 5.0 in stage 156.0 (TID 730)
in 934 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 5.0 in stage 162.0 (TID 757)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 26 to 10.183.229.123:55848
25/02/06 06:11:31 INFO TaskSetManager: Starting task 6.0 in stage 162.0 (TID 758)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 3.0 in stage 156.0 (TID 728)
in 1007 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 7.0 in stage 162.0 (TID 759)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 5.0 in stage 158.0 (TID 739)
in 1005 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:31 INFO TaskSetManager: Finished task 6.0 in stage 156.0 (TID 731)
in 1032 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:31 INFO TaskSchedulerImpl: Removed TaskSet 156.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:31 INFO DAGScheduler: ShuffleMapStage 156 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 127.333 s
25/02/06 06:11:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:31 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 158, ShuffleMapStage 260, ShuffleMapStage 341,
ShuffleMapStage 210, ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:31 INFO DAGScheduler: failed: Set()
25/02/06 06:11:31 INFO TaskSetManager: Starting task 8.0 in stage 162.0 (TID 760)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Starting task 0.0 in stage 164.0 (TID 761)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:31 INFO MemoryStore: Block broadcast_35 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:31 INFO TaskSetManager: Finished task 4.0 in stage 158.0 (TID 738)
in 1041 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:31 INFO TaskSetManager: Finished task 7.0 in stage 158.0 (TID 741)
in 853 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:31 INFO TaskSchedulerImpl: Removed TaskSet 158.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:31 INFO MemoryStore: Block broadcast_35_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:31 INFO DAGScheduler: ShuffleMapStage 158 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 127.336 s
25/02/06 06:11:31 INFO SparkContext: Created broadcast 35 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:31 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 204, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188,
ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 168, ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314,
ShuffleMapStage 162, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 160, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:31 INFO DAGScheduler: failed: Set()
25/02/06 06:11:31 INFO TaskSetManager: Starting task 1.0 in stage 164.0 (TID 762)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Starting task 2.0 in stage 164.0 (TID 763)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 8.0 in stage 160.0 (TID 751)
in 280 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 46 to 10.183.229.123:55848
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 46 to 10.183.246.72:39896
25/02/06 06:11:31 INFO TaskSetManager: Starting task 3.0 in stage 164.0 (TID 764)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 2.0 in stage 160.0 (TID 745)
in 822 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:31 INFO TaskSetManager: Starting task 4.0 in stage 164.0 (TID 765)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 0.0 in stage 160.0 (TID 743)
in 1030 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 46 to 10.183.233.191:55538
25/02/06 06:11:31 INFO TaskSetManager: Starting task 5.0 in stage 164.0 (TID 766)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 1.0 in stage 160.0 (TID 744)
in 1060 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:31 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099034) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099034)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:31 INFO TaskSetManager: Starting task 6.0 in stage 164.0 (TID 767)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:31 INFO TaskSetManager: Finished task 8.0 in stage 162.0 (TID 760)
in 377 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:31 INFO CodeGenerator: Code generated in 56.474699 ms
25/02/06 06:11:31 INFO CodeGenerator: Code generated in 53.305267 ms
25/02/06 06:11:31 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:31 INFO CodeGenerator: Code generated in 43.610999 ms
25/02/06 06:11:31 INFO CodeGenerator: Code generated in 62.651571 ms
25/02/06 06:11:31 INFO DAGScheduler: Registering RDD 1327
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 208
25/02/06 06:11:31 INFO DAGScheduler: Got map stage job 210
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:31 INFO DAGScheduler: Final stage: ShuffleMapStage 353
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
352)
25/02/06 06:11:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:31 INFO DAGScheduler: Submitting ShuffleMapStage 353
(MapPartitionsRDD[1327] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:31 INFO DAGScheduler: Registering RDD 1324
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 207
25/02/06 06:11:31 INFO DAGScheduler: Got map stage job 211
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:31 INFO DAGScheduler: Final stage: ShuffleMapStage 356
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
355)
25/02/06 06:11:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:31 INFO DAGScheduler: Submitting ShuffleMapStage 356
(MapPartitionsRDD[1324] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:31 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 353 (MapPartitionsRDD[1327] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:31 INFO TaskSchedulerImpl: Adding task set 353.0 with 6 tasks
resource profile 0
25/02/06 06:11:31 INFO TaskSetManager: TaskSet 353.0 using PreferredLocationsV1
25/02/06 06:11:31 INFO FairSchedulableBuilder: Added task set TaskSet_353.0 tasks
to pool 5400065285585087590
25/02/06 06:11:31 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 356 (MapPartitionsRDD[1324] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:31 INFO TaskSchedulerImpl: Adding task set 356.0 with 5 tasks
resource profile 0
25/02/06 06:11:31 INFO TaskSetManager: TaskSet 356.0 using PreferredLocationsV1
25/02/06 06:11:31 INFO FairSchedulableBuilder: Added task set TaskSet_356.0 tasks
to pool 5400065285585087590
25/02/06 06:11:31 INFO DAGScheduler: Registering RDD 1332
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 209
25/02/06 06:11:31 INFO DAGScheduler: Got map stage job 212
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:31 INFO DAGScheduler: Final stage: ShuffleMapStage 359
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
358)
25/02/06 06:11:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:31 INFO DAGScheduler: Submitting ShuffleMapStage 359
(MapPartitionsRDD[1332] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 46 to 10.183.249.197:36834
25/02/06 06:11:31 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 359 (MapPartitionsRDD[1332] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:31 INFO TaskSchedulerImpl: Adding task set 359.0 with 5 tasks
resource profile 0
25/02/06 06:11:31 INFO TaskSetManager: TaskSet 359.0 using PreferredLocationsV1
25/02/06 06:11:31 INFO FairSchedulableBuilder: Added task set TaskSet_359.0 tasks
to pool 5400065285585087590
25/02/06 06:11:31 INFO DAGScheduler: Registering RDD 1334
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 210
25/02/06 06:11:31 INFO DAGScheduler: Got map stage job 213
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:31 INFO DAGScheduler: Final stage: ShuffleMapStage 362
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
361)
25/02/06 06:11:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:31 INFO DAGScheduler: Submitting ShuffleMapStage 362
(MapPartitionsRDD[1334] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:31 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 362 (MapPartitionsRDD[1334] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:31 INFO TaskSchedulerImpl: Adding task set 362.0 with 6 tasks
resource profile 0
25/02/06 06:11:31 INFO TaskSetManager: TaskSet 362.0 using PreferredLocationsV1
25/02/06 06:11:31 INFO FairSchedulableBuilder: Added task set TaskSet_362.0 tasks
to pool 5400065285585087590
25/02/06 06:11:32 INFO TaskSetManager: Starting task 7.0 in stage 164.0 (TID 768)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 0.0 in stage 162.0 (TID 752)
in 771 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:32 INFO TaskSetManager: Finished task 4.0 in stage 160.0 (TID 747)
in 1001 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 8.0 in stage 164.0 (TID 769)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Starting task 0.0 in stage 166.0 (TID 770)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 5.0 in stage 160.0 (TID 748)
in 1012 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:32 INFO MemoryStore: Block broadcast_36 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:32 INFO MemoryStore: Block broadcast_36_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:32 INFO SparkContext: Created broadcast 36 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:32 INFO TaskSetManager: Starting task 1.0 in stage 166.0 (TID 771)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 2.0 in stage 162.0 (TID 754)
in 814 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 2.0 in stage 166.0 (TID 772)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 1.0 in stage 162.0 (TID 753)
in 826 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 44 to 10.183.232.45:60256
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 44 to 10.183.249.197:36834
25/02/06 06:11:32 INFO TaskSetManager: Starting task 3.0 in stage 166.0 (TID 773)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 4.0 in stage 162.0 (TID 756)
in 804 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 44 to 10.183.233.191:55538
25/02/06 06:11:32 INFO TaskSetManager: Starting task 4.0 in stage 166.0 (TID 774)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 7.0 in stage 160.0 (TID 750)
in 1017 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:32 INFO TaskSetManager: Finished task 0.0 in stage 164.0 (TID 761)
in 740 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 5.0 in stage 166.0 (TID 775)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 6.0 in stage 166.0 (TID 776)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 2.0 in stage 164.0 (TID 763)
in 733 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 44 to 10.183.246.72:39896
25/02/06 06:11:32 INFO TaskSetManager: Starting task 7.0 in stage 166.0 (TID 777)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 6.0 in stage 162.0 (TID 758)
in 804 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 44 to 10.183.229.123:55848
25/02/06 06:11:32 INFO TaskSetManager: Starting task 8.0 in stage 166.0 (TID 778)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 6.0 in stage 160.0 (TID 749)
in 1052 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:32 INFO TaskSetManager: Finished task 5.0 in stage 162.0 (TID 757)
in 921 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 0.0 in stage 168.0 (TID 779)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Starting task 1.0 in stage 168.0 (TID 780)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 1.0 in stage 164.0 (TID 762)
in 798 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:32 INFO MemoryStore: Block broadcast_37 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:32 INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:32 INFO SparkContext: Created broadcast 37 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 49 to 10.183.229.123:55848
25/02/06 06:11:32 INFO TaskSetManager: Starting task 2.0 in stage 168.0 (TID 781)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 7.0 in stage 162.0 (TID 759)
in 873 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 49 to 10.183.249.197:36834
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 49 to 10.183.232.45:60256
25/02/06 06:11:32 INFO TaskSetManager: Starting task 3.0 in stage 168.0 (TID 782)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 3.0 in stage 164.0 (TID 764)
in 688 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 4.0 in stage 168.0 (TID 783)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 3.0 in stage 160.0 (TID 746)
in 1313 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:32 INFO TaskSchedulerImpl: Removed TaskSet 160.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:32 INFO DAGScheduler: ShuffleMapStage 160 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.179 s
25/02/06 06:11:32 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:32 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 162,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:32 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:32 INFO DAGScheduler: failed: Set()
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 49 to 10.183.246.72:39896
25/02/06 06:11:32 INFO TaskSetManager: Starting task 5.0 in stage 168.0 (TID 784)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 8.0 in stage 164.0 (TID 769)
in 323 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:32 INFO TaskSetManager: Starting task 6.0 in stage 168.0 (TID 785)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:32 INFO TaskSetManager: Finished task 3.0 in stage 162.0 (TID 755)
in 1048 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:32 INFO TaskSchedulerImpl: Removed TaskSet 162.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:32 INFO DAGScheduler: ShuffleMapStage 162 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.212 s
25/02/06 06:11:32 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:32 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:32 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 164, ShuffleMapStage 266, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:32 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:32 INFO DAGScheduler: failed: Set()
25/02/06 06:11:32 INFO ShufflePartitionsUtil: For shuffle(121), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:32 INFO ShufflePartitionsUtil: For shuffle(122), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:32 INFO ShufflePartitionsUtil: For shuffle(123), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:32 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 49 to 10.183.233.191:55538
25/02/06 06:11:33 INFO TaskSetManager: Finished task 8.0 in stage 166.0 (TID 778)
in 872 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 7.0 in stage 168.0 (TID 786)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 8.0 in stage 168.0 (TID 787)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 0.0 in stage 170.0 (TID 788)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_38 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 1.0 in stage 170.0 (TID 789)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 2.0 in stage 170.0 (TID 790)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 3.0 in stage 170.0 (TID 791)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:33 INFO SparkContext: Created broadcast 38 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:33 INFO TaskSetManager: Starting task 4.0 in stage 170.0 (TID 792)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 5.0 in stage 170.0 (TID 793)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 6.0 in stage 170.0 (TID 794)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 7.0 in stage 170.0 (TID 795)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 7.0 in stage 166.0 (TID 777)
in 906 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 1.0 in stage 166.0 (TID 771)
in 1043 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 4.0 in stage 166.0 (TID 774)
in 925 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 4.0 in stage 164.0 (TID 765)
in 1417 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 7.0 in stage 164.0 (TID 768)
in 1123 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 3.0 in stage 166.0 (TID 773)
in 993 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 5.0 in stage 168.0 (TID 784)
in 757 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 6.0 in stage 166.0 (TID 776)
in 915 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 6.0 in stage 164.0 (TID 767)
in 1299 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 5.0 in stage 164.0 (TID 766)
in 1362 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:33 INFO TaskSchedulerImpl: Removed TaskSet 164.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:33 INFO TaskSetManager: Finished task 0.0 in stage 168.0 (TID 779)
in 863 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 2.0 in stage 166.0 (TID 772)
in 1042 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 3.0 in stage 168.0 (TID 782)
in 796 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 5.0 in stage 166.0 (TID 775)
in 926 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 0.0 in stage 166.0 (TID 770)
in 1054 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:33 INFO TaskSchedulerImpl: Removed TaskSet 166.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:33 INFO TaskSetManager: Starting task 8.0 in stage 170.0 (TID 796)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 0.0 in stage 172.0 (TID 797)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 1.0 in stage 172.0 (TID 798)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Starting task 2.0 in stage 172.0 (TID 799)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:33 INFO DAGScheduler: ShuffleMapStage 164 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.955 s
25/02/06 06:11:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:33 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 166, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:33 INFO DAGScheduler: failed: Set()
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 3.0 in stage 172.0 (TID 800)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 4.0 in stage 172.0 (TID 801)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_39 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:33 INFO SparkContext: Created broadcast 39 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 50 to 10.183.232.45:60256
25/02/06 06:11:33 INFO DAGScheduler: ShuffleMapStage 166 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.948 s
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 50 to 10.183.233.191:55538
25/02/06 06:11:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:33 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 168,
ShuffleMapStage 220, ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:33 INFO DAGScheduler: failed: Set()
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 7 to 10.183.229.123:55848
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 50 to 10.183.246.72:39896
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 50 to 10.183.249.197:36834
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 7 to 10.183.249.197:36834
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 7 to 10.183.246.72:39896
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:33 INFO TaskSetManager: Starting task 5.0 in stage 172.0 (TID 802)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 2.0 in stage 168.0 (TID 781)
in 956 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 6.0 in stage 172.0 (TID 803)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 4.0 in stage 168.0 (TID 783)
in 935 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 7 to 10.183.232.45:60256
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:33 INFO TaskSetManager: Starting task 7.0 in stage 172.0 (TID 804)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 1.0 in stage 168.0 (TID 780)
in 1138 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 8.0 in stage 172.0 (TID 805)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 8.0 in stage 168.0 (TID 787)
in 312 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 0.0 in stage 174.0 (TID 806)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 6.0 in stage 168.0 (TID 785)
in 1127 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_40 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO MemoryStore: Block broadcast_40_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:33 INFO SparkContext: Created broadcast 40 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 8.0 in stage 170.0 (TID 796)
in 400 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 1.0 in stage 174.0 (TID 807)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 30 to 10.183.229.123:55848
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 30 to 10.183.249.197:36834
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:33 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(204,1738822288311,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@7fa913d1, org.apache.spark.scheduler.StageInfo@6dbb0408,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@3d79da8b),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 1.228486829s.
25/02/06 06:11:33 INFO TaskSetManager: Starting task 2.0 in stage 174.0 (TID 808)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 8.0 in stage 172.0 (TID 805)
in 340 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 30 to 10.183.232.45:60256
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:33 INFO TaskSetManager: Starting task 3.0 in stage 174.0 (TID 809)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 2.0 in stage 172.0 (TID 799)
in 765 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:33 INFO TaskSetManager: Finished task 2.0 in stage 170.0 (TID 790)
in 784 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 4.0 in stage 174.0 (TID 810)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:33 INFO TaskSetManager: Finished task 1.0 in stage 170.0 (TID 789)
in 793 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:33 INFO TaskSetManager: Starting task 5.0 in stage 174.0 (TID 811)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:33 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 30 to 10.183.246.72:39896
25/02/06 06:11:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:34 INFO TaskSetManager: Starting task 6.0 in stage 174.0 (TID 812)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 7.0 in stage 170.0 (TID 795)
in 848 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:34 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100115) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100115)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:34 INFO TaskSetManager: Finished task 7.0 in stage 168.0 (TID 786)
in 883 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:34 INFO TaskSchedulerImpl: Removed TaskSet 168.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:34 INFO DAGScheduler: ShuffleMapStage 168 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.105 s
25/02/06 06:11:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:34 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 176,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 172, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 220,
ShuffleMapStage 170, ShuffleMapStage 314, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:34 INFO DAGScheduler: failed: Set()
25/02/06 06:11:34 INFO TaskSetManager: Starting task 7.0 in stage 174.0 (TID 813)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 8.0 in stage 174.0 (TID 814)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 4.0 in stage 170.0 (TID 792)
in 895 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 30 to 10.183.233.191:55538
25/02/06 06:11:34 INFO CodeGenerator: Code generated in 56.748007 ms
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:34 INFO CodeGenerator: Code generated in 40.28754 ms
25/02/06 06:11:34 INFO DAGScheduler: Registering RDD 1339
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 211
25/02/06 06:11:34 INFO DAGScheduler: Got map stage job 214
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:34 INFO DAGScheduler: Final stage: ShuffleMapStage 365
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:34 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
364)
25/02/06 06:11:34 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:34 INFO DAGScheduler: Submitting ShuffleMapStage 365
(MapPartitionsRDD[1339] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:34 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 365 (MapPartitionsRDD[1339] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:34 INFO TaskSchedulerImpl: Adding task set 365.0 with 6 tasks
resource profile 0
25/02/06 06:11:34 INFO TaskSetManager: TaskSet 365.0 using PreferredLocationsV1
25/02/06 06:11:34 INFO FairSchedulableBuilder: Added task set TaskSet_365.0 tasks
to pool 5400065285585087590
25/02/06 06:11:34 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:34 INFO TaskSetManager: Finished task 6.0 in stage 170.0 (TID 794)
in 937 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 0.0 in stage 176.0 (TID 815)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_41 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO CodeGenerator: Code generated in 61.429586 ms
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:34 INFO SparkContext: Created broadcast 41 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:34 INFO TaskSetManager: Starting task 1.0 in stage 176.0 (TID 816)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 3.0 in stage 170.0 (TID 791)
in 950 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 2.0 in stage 176.0 (TID 817)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 5.0 in stage 172.0 (TID 802)
in 809 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:34 INFO DAGScheduler: Registering RDD 1346
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 212
25/02/06 06:11:34 INFO DAGScheduler: Got map stage job 215
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:34 INFO DAGScheduler: Final stage: ShuffleMapStage 368
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:34 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
367)
25/02/06 06:11:34 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:34 INFO DAGScheduler: Submitting ShuffleMapStage 368
(MapPartitionsRDD[1346] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 368 (MapPartitionsRDD[1346] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:34 INFO TaskSchedulerImpl: Adding task set 368.0 with 5 tasks
resource profile 0
25/02/06 06:11:34 INFO TaskSetManager: TaskSet 368.0 using PreferredLocationsV1
25/02/06 06:11:34 INFO FairSchedulableBuilder: Added task set TaskSet_368.0 tasks
to pool 5400065285585087590
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 23 to 10.183.232.45:60256
25/02/06 06:11:34 INFO DAGScheduler: Registering RDD 1349
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 213
25/02/06 06:11:34 INFO DAGScheduler: Got map stage job 216
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:34 INFO DAGScheduler: Final stage: ShuffleMapStage 371
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:34 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
370)
25/02/06 06:11:34 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:34 INFO DAGScheduler: Submitting ShuffleMapStage 371
(MapPartitionsRDD[1349] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:34 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 371 (MapPartitionsRDD[1349] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:34 INFO TaskSchedulerImpl: Adding task set 371.0 with 6 tasks
resource profile 0
25/02/06 06:11:34 INFO TaskSetManager: TaskSet 371.0 using PreferredLocationsV1
25/02/06 06:11:34 INFO FairSchedulableBuilder: Added task set TaskSet_371.0 tasks
to pool 5400065285585087590
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 23 to 10.183.233.191:55538
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 23 to 10.183.246.72:39896
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:34 INFO TaskSetManager: Finished task 0.0 in stage 172.0 (TID 797)
in 1000 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 3.0 in stage 176.0 (TID 818)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 1.0 in stage 172.0 (TID 798)
in 1002 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 4.0 in stage 176.0 (TID 819)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 5.0 in stage 170.0 (TID 793)
in 1014 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 5.0 in stage 176.0 (TID 820)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 23 to 10.183.249.197:36834
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:34 INFO TaskSetManager: Starting task 6.0 in stage 176.0 (TID 821)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 3.0 in stage 172.0 (TID 800)
in 1054 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 7.0 in stage 176.0 (TID 822)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 6.0 in stage 172.0 (TID 803)
in 932 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 8.0 in stage 176.0 (TID 823)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 0.0 in stage 170.0 (TID 788)
in 1115 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:34 INFO TaskSchedulerImpl: Removed TaskSet 170.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:34 INFO DAGScheduler: ShuffleMapStage 170 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 126.035 s
25/02/06 06:11:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:34 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 172, ShuffleMapStage 224,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:34 INFO DAGScheduler: failed: Set()
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 23 to 10.183.229.123:55848
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:34 INFO TaskSetManager: Starting task 0.0 in stage 178.0 (TID 824)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 4.0 in stage 172.0 (TID 801)
in 1151 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_42 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_42_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:34 INFO SparkContext: Created broadcast 42 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 40 to 10.183.229.123:55848
25/02/06 06:11:34 INFO TaskSetManager: Starting task 1.0 in stage 178.0 (TID 825)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 1.0 in stage 174.0 (TID 807)
in 819 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 40 to 10.183.249.197:36834
25/02/06 06:11:34 INFO TaskSetManager: Finished task 7.0 in stage 172.0 (TID 804)
in 1010 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:34 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:34 INFO DAGScheduler: ShuffleMapStage 172 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 124.834 s
25/02/06 06:11:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:34 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 174, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:34 INFO DAGScheduler: failed: Set()
25/02/06 06:11:34 INFO TaskSetManager: Starting task 2.0 in stage 178.0 (TID 826)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 8.0 in stage 174.0 (TID 814)
in 457 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 3.0 in stage 178.0 (TID 827)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 40 to 10.183.233.191:55538
25/02/06 06:11:34 INFO TaskSetManager: Starting task 4.0 in stage 178.0 (TID 828)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 8.0 in stage 176.0 (TID 823)
in 292 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:34 INFO ShufflePartitionsUtil: For shuffle(124), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 5.0 in stage 178.0 (TID 829)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 0.0 in stage 174.0 (TID 806)
in 1021 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:34 INFO ShufflePartitionsUtil: For shuffle(125), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:34 INFO ShufflePartitionsUtil: For shuffle(126), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 40 to 10.183.232.45:60256
25/02/06 06:11:34 INFO ShufflePartitionsUtil: For shuffle(127), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:34 INFO TaskSetManager: Starting task 6.0 in stage 178.0 (TID 830)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 2.0 in stage 174.0 (TID 808)
in 797 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 7.0 in stage 178.0 (TID 831)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 4.0 in stage 174.0 (TID 810)
in 790 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 40 to 10.183.246.72:39896
25/02/06 06:11:34 INFO TaskSetManager: Finished task 7.0 in stage 174.0 (TID 813)
in 718 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 8.0 in stage 178.0 (TID 832)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 5.0 in stage 174.0 (TID 811)
in 830 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 0.0 in stage 180.0 (TID 833)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_43 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:34 INFO SparkContext: Created broadcast 43 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 39 to 10.183.246.72:39896
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:34 INFO TaskSetManager: Starting task 1.0 in stage 180.0 (TID 834)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 3.0 in stage 174.0 (TID 809)
in 919 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 39 to 10.183.249.197:36834
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:34 INFO TaskSetManager: Starting task 2.0 in stage 180.0 (TID 835)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 6.0 in stage 174.0 (TID 812)
in 885 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:34 INFO TaskSchedulerImpl: Removed TaskSet 174.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:34 INFO DAGScheduler: ShuffleMapStage 174 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 123.106 s
25/02/06 06:11:34 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:34 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 176, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 178, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 180, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266,
ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:34 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:34 INFO DAGScheduler: failed: Set()
25/02/06 06:11:34 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 39 to 10.183.233.191:55538
25/02/06 06:11:34 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:34 INFO TaskSetManager: Starting task 3.0 in stage 180.0 (TID 836)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 1.0 in stage 176.0 (TID 816)
in 862 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:34 INFO TaskSetManager: Starting task 4.0 in stage 180.0 (TID 837)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:34 INFO TaskSetManager: Finished task 0.0 in stage 176.0 (TID 815)
in 885 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:35 INFO TaskSetManager: Starting task 5.0 in stage 180.0 (TID 838)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Finished task 8.0 in stage 178.0 (TID 832)
in 266 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 39 to 10.183.232.45:60256
25/02/06 06:11:35 INFO TaskSetManager: Starting task 6.0 in stage 180.0 (TID 839)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Finished task 2.0 in stage 176.0 (TID 817)
in 916 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:35 INFO TaskSetManager: Finished task 5.0 in stage 176.0 (TID 820)
in 1587 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 0.0 in stage 178.0 (TID 824)
in 1441 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:35 INFO TaskSetManager: Starting task 7.0 in stage 180.0 (TID 840)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 8.0 in stage 180.0 (TID 841)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Finished task 4.0 in stage 176.0 (TID 819)
in 1593 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 5.0 in stage 178.0 (TID 829)
in 1208 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 2.0 in stage 178.0 (TID 826)
in 1313 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 6.0 in stage 176.0 (TID 821)
in 1544 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 7.0 in stage 178.0 (TID 831)
in 1040 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 0.0 in stage 180.0 (TID 833)
in 994 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 3.0 in stage 178.0 (TID 827)
in 1265 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 7.0 in stage 176.0 (TID 822)
in 1531 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 4.0 in stage 180.0 (TID 837)
in 791 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 3.0 in stage 176.0 (TID 818)
in 1603 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:35 INFO TaskSchedulerImpl: Removed TaskSet 176.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:35 INFO TaskSetManager: Finished task 1.0 in stage 178.0 (TID 825)
in 1384 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:35 INFO DAGScheduler: ShuffleMapStage 176 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 120.730 s
25/02/06 06:11:35 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:35 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 178,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 180, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:35 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:35 INFO TaskSetManager: Finished task 1.0 in stage 180.0 (TID 834)
in 922 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:35 INFO DAGScheduler: failed: Set()
25/02/06 06:11:35 INFO TaskSetManager: Finished task 4.0 in stage 178.0 (TID 828)
in 1229 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 2.0 in stage 180.0 (TID 835)
in 888 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 6.0 in stage 180.0 (TID 839)
in 756 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:35 INFO TaskSetManager: Finished task 3.0 in stage 180.0 (TID 836)
in 817 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:35 INFO TaskSetManager: Starting task 0.0 in stage 182.0 (TID 842)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 1.0 in stage 182.0 (TID 843)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 2.0 in stage 182.0 (TID 844)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 3.0 in stage 182.0 (TID 845)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:35 INFO MemoryStore: Block broadcast_44 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:35 INFO TaskSetManager: Starting task 4.0 in stage 182.0 (TID 846)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 5.0 in stage 182.0 (TID 847)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 6.0 in stage 182.0 (TID 848)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:35 INFO MemoryStore: Block broadcast_44_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:35 INFO TaskSetManager: Starting task 7.0 in stage 182.0 (TID 849)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 8.0 in stage 182.0 (TID 850)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 0.0 in stage 184.0 (TID 851)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 1.0 in stage 184.0 (TID 852)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Finished task 6.0 in stage 178.0 (TID 830)
in 1198 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:35 INFO TaskSchedulerImpl: Removed TaskSet 178.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:35 INFO TaskSetManager: Starting task 2.0 in stage 184.0 (TID 853)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 3.0 in stage 184.0 (TID 854)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 4.0 in stage 184.0 (TID 855)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 39 to 10.183.229.123:55848
25/02/06 06:11:35 INFO DAGScheduler: ShuffleMapStage 178 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 119.163 s
25/02/06 06:11:35 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:35 INFO TaskSetManager: Starting task 5.0 in stage 184.0 (TID 856)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 6.0 in stage 184.0 (TID 857)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:35 INFO TaskSetManager: Starting task 7.0 in stage 184.0 (TID 858)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:35 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 194,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 186, ShuffleMapStage 238, ShuffleMapStage 188, ShuffleMapStage 311,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 182,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234, ShuffleMapStage 184,
ShuffleMapStage 286, ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 180,
ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:35 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:35 INFO DAGScheduler: failed: Set()
25/02/06 06:11:35 INFO TaskSetManager: Starting task 8.0 in stage 184.0 (TID 859)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:35 INFO TaskSetManager: Finished task 5.0 in stage 180.0 (TID 838)
in 789 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:35 INFO SparkContext: Created broadcast 44 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:35 INFO MemoryStore: Block broadcast_45 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:35 INFO MemoryStore: Block broadcast_45_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:35 INFO SparkContext: Created broadcast 45 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 53 to 10.183.246.72:39896
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 53 to 10.183.249.197:36834
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 11 to 10.183.246.72:39896
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 11 to 10.183.232.45:60256
25/02/06 06:11:35 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 11 to 10.183.233.191:55538
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 53 to 10.183.229.123:55848
25/02/06 06:11:35 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:35 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100082) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100082)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:35 INFO CodeGenerator: Code generated in 41.430264 ms
25/02/06 06:11:36 INFO CodeGenerator: Code generated in 50.565627 ms
25/02/06 06:11:36 INFO DAGScheduler: Registering RDD 1354
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 214
25/02/06 06:11:36 INFO DAGScheduler: Got map stage job 217
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:36 INFO DAGScheduler: Final stage: ShuffleMapStage 374
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
373)
25/02/06 06:11:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:36 INFO DAGScheduler: Submitting ShuffleMapStage 374
(MapPartitionsRDD[1354] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:36 INFO CodeGenerator: Code generated in 68.252712 ms
25/02/06 06:11:36 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 374 (MapPartitionsRDD[1354] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:36 INFO TaskSchedulerImpl: Adding task set 374.0 with 5 tasks
resource profile 0
25/02/06 06:11:36 INFO TaskSetManager: TaskSet 374.0 using PreferredLocationsV1
25/02/06 06:11:36 INFO FairSchedulableBuilder: Added task set TaskSet_374.0 tasks
to pool 5400065285585087590
25/02/06 06:11:36 INFO CodeGenerator: Code generated in 65.722302 ms
25/02/06 06:11:36 INFO DAGScheduler: Registering RDD 1359
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 215
25/02/06 06:11:36 INFO DAGScheduler: Got map stage job 218
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:36 INFO DAGScheduler: Final stage: ShuffleMapStage 377
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
376)
25/02/06 06:11:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:36 INFO DAGScheduler: Submitting ShuffleMapStage 377
(MapPartitionsRDD[1359] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:36 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 377 (MapPartitionsRDD[1359] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:36 INFO TaskSchedulerImpl: Adding task set 377.0 with 6 tasks
resource profile 0
25/02/06 06:11:36 INFO TaskSetManager: TaskSet 377.0 using PreferredLocationsV1
25/02/06 06:11:36 INFO FairSchedulableBuilder: Added task set TaskSet_377.0 tasks
to pool 5400065285585087590
25/02/06 06:11:36 INFO DAGScheduler: Registering RDD 1365
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 216
25/02/06 06:11:36 INFO DAGScheduler: Got map stage job 219
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:36 INFO DAGScheduler: Final stage: ShuffleMapStage 380
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
379)
25/02/06 06:11:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:36 INFO DAGScheduler: Submitting ShuffleMapStage 380
(MapPartitionsRDD[1365] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:36 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 380 (MapPartitionsRDD[1365] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:36 INFO TaskSchedulerImpl: Adding task set 380.0 with 6 tasks
resource profile 0
25/02/06 06:11:36 INFO TaskSetManager: TaskSet 380.0 using PreferredLocationsV1
25/02/06 06:11:36 INFO FairSchedulableBuilder: Added task set TaskSet_380.0 tasks
to pool 5400065285585087590
25/02/06 06:11:36 INFO DAGScheduler: Registering RDD 1369
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 217
25/02/06 06:11:36 INFO DAGScheduler: Got map stage job 220
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:36 INFO DAGScheduler: Final stage: ShuffleMapStage 383
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
382)
25/02/06 06:11:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:36 INFO DAGScheduler: Submitting ShuffleMapStage 383
(MapPartitionsRDD[1369] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:36 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 383 (MapPartitionsRDD[1369] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:36 INFO TaskSchedulerImpl: Adding task set 383.0 with 6 tasks
resource profile 0
25/02/06 06:11:36 INFO TaskSetManager: TaskSet 383.0 using PreferredLocationsV1
25/02/06 06:11:36 INFO FairSchedulableBuilder: Added task set TaskSet_383.0 tasks
to pool 5400065285585087590
25/02/06 06:11:36 INFO TaskSetManager: Finished task 8.0 in stage 184.0 (TID 859)
in 495 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 0.0 in stage 186.0 (TID 860)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:36 INFO MemoryStore: Block broadcast_46 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:36 INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:36 INFO SparkContext: Created broadcast 46 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 22 to 10.183.232.45:60256
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:36 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(207,1738822290802,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@53f0d432, org.apache.spark.scheduler.StageInfo@64f06206,
org.apache.spark.scheduler.StageInfo@515ea1d4,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 1.341114181s.
25/02/06 06:11:36 INFO TaskSetManager: Starting task 1.0 in stage 186.0 (TID 861)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 8.0 in stage 182.0 (TID 850)
in 580 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(133), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 2.0 in stage 186.0 (TID 862)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 8.0 in stage 180.0 (TID 841)
in 616 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(131), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 22 to 10.183.246.72:39896
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(132), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 22 to 10.183.229.123:55848
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(130), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(128), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO ShufflePartitionsUtil: For shuffle(129), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:36 INFO TaskSetManager: Starting task 3.0 in stage 186.0 (TID 863)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 7.0 in stage 180.0 (TID 840)
in 757 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:36 INFO TaskSchedulerImpl: Removed TaskSet 180.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:36 INFO DAGScheduler: ShuffleMapStage 180 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 117.945 s
25/02/06 06:11:36 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:36 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 184, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:36 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:36 INFO DAGScheduler: failed: Set()
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 22 to 10.183.233.191:55538
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:36 INFO TaskSetManager: Starting task 4.0 in stage 186.0 (TID 864)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 2.0 in stage 182.0 (TID 844)
in 775 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 22 to 10.183.249.197:36834
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:36 INFO TaskSetManager: Starting task 5.0 in stage 186.0 (TID 865)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 4.0 in stage 184.0 (TID 855)
in 797 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 6.0 in stage 186.0 (TID 866)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 7.0 in stage 182.0 (TID 849)
in 803 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 7.0 in stage 186.0 (TID 867)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 2.0 in stage 184.0 (TID 853)
in 850 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:36 INFO TaskSetManager: Finished task 6.0 in stage 184.0 (TID 857)
in 858 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 8.0 in stage 186.0 (TID 868)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 0.0 in stage 182.0 (TID 842)
in 925 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 0.0 in stage 188.0 (TID 869)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:36 INFO MemoryStore: Block broadcast_47 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:36 INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:36 INFO SparkContext: Created broadcast 47 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 34 to 10.183.249.197:36834
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:36 INFO TaskSetManager: Starting task 1.0 in stage 188.0 (TID 870)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 5.0 in stage 182.0 (TID 847)
in 969 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:36 INFO TaskSetManager: Finished task 0.0 in stage 184.0 (TID 851)
in 976 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 2.0 in stage 188.0 (TID 871)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO TaskSetManager: Finished task 5.0 in stage 184.0 (TID 856)
in 978 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 3.0 in stage 188.0 (TID 872)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 34 to 10.183.232.45:60256
25/02/06 06:11:36 INFO TaskSetManager: Finished task 1.0 in stage 182.0 (TID 843)
in 1006 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 34 to 10.183.246.72:39896
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 34 to 10.183.229.123:55848
25/02/06 06:11:36 INFO TaskSetManager: Starting task 4.0 in stage 188.0 (TID 873)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:36 INFO TaskSetManager: Finished task 3.0 in stage 182.0 (TID 845)
in 1028 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 5.0 in stage 188.0 (TID 874)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Starting task 6.0 in stage 188.0 (TID 875)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 6.0 in stage 182.0 (TID 848)
in 1041 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 7.0 in stage 188.0 (TID 876)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 3.0 in stage 184.0 (TID 854)
in 1173 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 8.0 in stage 188.0 (TID 877)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:11:36 INFO TaskSetManager: Finished task 8.0 in stage 186.0 (TID 868)
in 314 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:36 INFO TaskSetManager: Finished task 7.0 in stage 184.0 (TID 858)
in 1175 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:36 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:36 INFO TaskSetManager: Starting task 0.0 in stage 190.0 (TID 878)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:36 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 34 to 10.183.233.191:55538
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:37 INFO TaskSetManager: Starting task 0.0 in stage 192.0 (TID 879)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 1.0 in stage 184.0 (TID 852)
in 1208 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:37 INFO TaskSchedulerImpl: Removed TaskSet 184.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:37 INFO DAGScheduler: ShuffleMapStage 184 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 114.543 s
25/02/06 06:11:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:37 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 182, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266,
ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:37 INFO DAGScheduler: failed: Set()
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_48 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:37 INFO SparkContext: Created broadcast 48 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 31 to 10.183.246.72:39896
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:37 INFO TaskSetManager: Starting task 1.0 in stage 192.0 (TID 880)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 0.0 in stage 186.0 (TID 860)
in 745 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 31 to 10.183.232.45:60256
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 18.993791 ms
25/02/06 06:11:37 INFO TaskSetManager: Starting task 2.0 in stage 192.0 (TID 881)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 4.0 in stage 182.0 (TID 846)
in 1348 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:37 INFO TaskSchedulerImpl: Removed TaskSet 182.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:37 INFO DAGScheduler: ShuffleMapStage 182 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 117.060 s
25/02/06 06:11:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:37 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 196,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 198,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 186, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:37 INFO DAGScheduler: failed: Set()
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 26.846726 ms
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1374
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 218
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 221
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 386
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
385)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 386
(MapPartitionsRDD[1374] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 386 (MapPartitionsRDD[1374] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 386.0 with 6 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 386.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 31 to 10.183.229.123:55848
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_386.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 37.547063 ms
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1379
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 219
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 222
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 389
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
388)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 389
(MapPartitionsRDD[1379] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 389 (MapPartitionsRDD[1379] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 389.0 with 6 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 389.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_389.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098395) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098395)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1384
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 220
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 223
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 392
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
391)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 392
(MapPartitionsRDD[1384] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 392 (MapPartitionsRDD[1384] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 392.0 with 5 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 392.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_392.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 INFO TaskSetManager: Finished task 1.0 in stage 186.0 (TID 861)
in 847 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 3.0 in stage 192.0 (TID 882)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 50.292864 ms
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 44.64222 ms
25/02/06 06:11:37 INFO CodeGenerator: Code generated in 53.334704 ms
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1392
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 221
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 224
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 395
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
394)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 395
(MapPartitionsRDD[1392] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 395 (MapPartitionsRDD[1392] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 395.0 with 6 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 395.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_395.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1399
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 223
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 225
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 398
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
397)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 398
(MapPartitionsRDD[1399] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Registering RDD 1397
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 222
25/02/06 06:11:37 INFO DAGScheduler: Got map stage job 226
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:37 INFO DAGScheduler: Final stage: ShuffleMapStage 401
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
400)
25/02/06 06:11:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:37 INFO TaskSetManager: Starting task 4.0 in stage 192.0 (TID 883)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 8.0 in stage 188.0 (TID 877)
in 329 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:37 INFO DAGScheduler: Submitting ShuffleMapStage 401
(MapPartitionsRDD[1397] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:37 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 398 (MapPartitionsRDD[1399] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 398.0 with 5 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 398.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_398.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 401 (MapPartitionsRDD[1397] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:37 INFO TaskSchedulerImpl: Adding task set 401.0 with 6 tasks
resource profile 0
25/02/06 06:11:37 INFO TaskSetManager: TaskSet 401.0 using PreferredLocationsV1
25/02/06 06:11:37 INFO FairSchedulableBuilder: Added task set TaskSet_401.0 tasks
to pool 5400065285585087590
25/02/06 06:11:37 INFO TaskSetManager: Starting task 5.0 in stage 192.0 (TID 884)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 5.0 in stage 186.0 (TID 865)
in 744 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:37 INFO TaskSetManager: Finished task 2.0 in stage 186.0 (TID 862)
in 952 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:37 INFO TaskSetManager: Finished task 3.0 in stage 186.0 (TID 863)
in 814 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 6.0 in stage 192.0 (TID 885)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Starting task 7.0 in stage 192.0 (TID 886)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 31 to 10.183.233.191:55538
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:37 INFO TaskSetManager: Finished task 4.0 in stage 186.0 (TID 864)
in 897 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 8.0 in stage 192.0 (TID 887)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 31 to 10.183.249.197:36834
25/02/06 06:11:37 INFO TaskSetManager: Starting task 0.0 in stage 194.0 (TID 888)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 6.0 in stage 186.0 (TID 866)
in 894 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_49 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_49_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:37 INFO SparkContext: Created broadcast 49 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 27 to 10.183.246.72:39896
25/02/06 06:11:37 INFO TaskSetManager: Finished task 7.0 in stage 186.0 (TID 867)
in 933 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:37 INFO TaskSchedulerImpl: Removed TaskSet 186.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:37 INFO DAGScheduler: ShuffleMapStage 186 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 111.996 s
25/02/06 06:11:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:37 INFO TaskSetManager: Starting task 1.0 in stage 194.0 (TID 889)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:37 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238,
ShuffleMapStage 188, ShuffleMapStage 311, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 386, ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:37 INFO DAGScheduler: failed: Set()
25/02/06 06:11:37 INFO TaskSetManager: Starting task 2.0 in stage 194.0 (TID 890)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 2.0 in stage 188.0 (TID 871)
in 816 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO TaskSetManager: Finished task 1.0 in stage 188.0 (TID 870)
in 828 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 3.0 in stage 194.0 (TID 891)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 27 to 10.183.233.191:55538
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 27 to 10.183.229.123:55848
25/02/06 06:11:37 INFO TaskSetManager: Starting task 4.0 in stage 194.0 (TID 892)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 5.0 in stage 188.0 (TID 874)
in 790 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 5.0 in stage 194.0 (TID 893)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 6.0 in stage 188.0 (TID 875)
in 782 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 27 to 10.183.249.197:36834
25/02/06 06:11:37 INFO TaskSetManager: Finished task 4.0 in stage 188.0 (TID 873)
in 882 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 6.0 in stage 194.0 (TID 894)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 0.0 in stage 188.0 (TID 869)
in 978 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 7.0 in stage 194.0 (TID 895)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:37 INFO ShufflePartitionsUtil: For shuffle(136), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:37 INFO ShufflePartitionsUtil: For shuffle(134), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:37 INFO TaskSetManager: Finished task 7.0 in stage 188.0 (TID 876)
in 779 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:37 INFO TaskSetManager: Starting task 8.0 in stage 194.0 (TID 896)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:37 INFO ShufflePartitionsUtil: For shuffle(135), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:37 INFO TaskSetManager: Starting task 0.0 in stage 196.0 (TID 897)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:37 INFO TaskSetManager: Finished task 8.0 in stage 192.0 (TID 887)
in 355 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:37 INFO TaskSetManager: Finished task 3.0 in stage 188.0 (TID 872)
in 1036 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:37 INFO TaskSchedulerImpl: Removed TaskSet 188.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_50 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO DAGScheduler: ShuffleMapStage 188 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 110.522 s
25/02/06 06:11:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:37 INFO TaskSetManager: Starting task 1.0 in stage 196.0 (TID 898)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:37 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 202,
ShuffleMapStage 194, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 192, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238,
ShuffleMapStage 311, ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:37 INFO DAGScheduler: failed: Set()
25/02/06 06:11:37 INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:37 INFO SparkContext: Created broadcast 50 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 58 to 10.183.232.45:60256
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 58 to 10.183.249.197:36834
25/02/06 06:11:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:38 INFO TaskSetManager: Starting task 2.0 in stage 196.0 (TID 899)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Starting task 3.0 in stage 196.0 (TID 900)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Starting task 4.0 in stage 196.0 (TID 901)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 1.0 in stage 192.0 (TID 880)
in 1163 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 2.0 in stage 192.0 (TID 881)
in 1061 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 3.0 in stage 192.0 (TID 882)
in 982 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:38 INFO TaskSetManager: Starting task 5.0 in stage 196.0 (TID 902)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Starting task 6.0 in stage 196.0 (TID 903)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Starting task 7.0 in stage 196.0 (TID 904)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 0.0 in stage 192.0 (TID 879)
in 1209 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 0.0 in stage 194.0 (TID 888)
in 721 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 8.0 in stage 194.0 (TID 896)
in 466 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 2.0 in stage 194.0 (TID 890)
in 629 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 58 to 10.183.246.72:39896
25/02/06 06:11:38 INFO TaskSetManager: Starting task 8.0 in stage 196.0 (TID 905)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 58 to 10.183.233.191:55538
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:38 INFO TaskSetManager: Finished task 5.0 in stage 192.0 (TID 884)
in 895 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:38 INFO TaskSetManager: Starting task 0.0 in stage 198.0 (TID 906)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:38 INFO MemoryStore: Block broadcast_51 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:38 INFO MemoryStore: Block broadcast_51_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:38 INFO SparkContext: Created broadcast 51 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 58 to 10.183.229.123:55848
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 4.0 in stage 192.0 (TID 883)
in 954 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:38 INFO TaskSetManager: Starting task 1.0 in stage 198.0 (TID 907)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 60 to 10.183.233.191:55538
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 60 to 10.183.232.45:60256
25/02/06 06:11:38 INFO TaskSetManager: Starting task 2.0 in stage 198.0 (TID 908)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 7.0 in stage 192.0 (TID 886)
in 950 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:38 INFO TaskSetManager: Starting task 3.0 in stage 198.0 (TID 909)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 1.0 in stage 194.0 (TID 889)
in 751 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.232.45:60256
25/02/06 06:11:38 INFO TaskSetManager: Starting task 4.0 in stage 198.0 (TID 910)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 4.0 in stage 194.0 (TID 892)
in 746 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 60 to 10.183.249.197:36834
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:38 INFO TaskSetManager: Starting task 5.0 in stage 198.0 (TID 911)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 6.0 in stage 194.0 (TID 894)
in 759 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 5.0 in stage 194.0 (TID 893)
in 891 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:38 INFO TaskSetManager: Starting task 6.0 in stage 198.0 (TID 912)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 60 to 10.183.229.123:55848
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:38 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098277) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098277)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:38 INFO TaskSetManager: Starting task 7.0 in stage 198.0 (TID 913)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 7.0 in stage 194.0 (TID 895)
in 872 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:38 INFO CodeGenerator: Code generated in 41.802296 ms
25/02/06 06:11:38 INFO DAGScheduler: Registering RDD 1404
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 224
25/02/06 06:11:38 INFO DAGScheduler: Got map stage job 227
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:38 INFO DAGScheduler: Final stage: ShuffleMapStage 404
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:38 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
403)
25/02/06 06:11:38 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:38 INFO DAGScheduler: Submitting ShuffleMapStage 404
(MapPartitionsRDD[1404] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:38 INFO TaskSetManager: Starting task 8.0 in stage 198.0 (TID 914)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 6.0 in stage 192.0 (TID 885)
in 1264 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:38 INFO TaskSchedulerImpl: Removed TaskSet 192.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:38 INFO DAGScheduler: ShuffleMapStage 192 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 107.772 s
25/02/06 06:11:38 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:38 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 202, ShuffleMapStage 194, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238,
ShuffleMapStage 311, ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:38 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:38 INFO DAGScheduler: failed: Set()
25/02/06 06:11:38 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 404 (MapPartitionsRDD[1404] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:38 INFO TaskSchedulerImpl: Adding task set 404.0 with 6 tasks
resource profile 0
25/02/06 06:11:38 INFO TaskSetManager: TaskSet 404.0 using PreferredLocationsV1
25/02/06 06:11:38 INFO FairSchedulableBuilder: Added task set TaskSet_404.0 tasks
to pool 5400065285585087590
25/02/06 06:11:38 INFO TaskSetManager: Finished task 3.0 in stage 194.0 (TID 891)
in 1025 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:38 INFO TaskSchedulerImpl: Removed TaskSet 194.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:38 INFO TaskSetManager: Starting task 0.0 in stage 200.0 (TID 915)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:38 INFO MemoryStore: Block broadcast_52 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:38 INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in
memory (estimated size 38.6 KiB, free 20.2 GiB)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.232.44:37705 (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:11:38 INFO DAGScheduler: ShuffleMapStage 194 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 106.594 s
25/02/06 06:11:38 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:38 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200, ShuffleMapStage 302,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 202, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 246,
ShuffleMapStage 196, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238, ShuffleMapStage 311,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 386, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:38 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:38 INFO DAGScheduler: failed: Set()
25/02/06 06:11:38 INFO SparkContext: Created broadcast 52 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:38 INFO CodeGenerator: Code generated in 69.110165 ms
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.229.123:38441 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO CodeGenerator: Code generated in 79.256612 ms
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 59 to 10.183.229.123:55848
25/02/06 06:11:38 INFO DAGScheduler: Registering RDD 1409
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 225
25/02/06 06:11:38 INFO DAGScheduler: Got map stage job 228
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:38 INFO DAGScheduler: Final stage: ShuffleMapStage 407
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:38 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
406)
25/02/06 06:11:38 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:38 INFO DAGScheduler: Submitting ShuffleMapStage 407
(MapPartitionsRDD[1409] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:38 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 407 (MapPartitionsRDD[1409] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:38 INFO TaskSchedulerImpl: Adding task set 407.0 with 6 tasks
resource profile 0
25/02/06 06:11:38 INFO TaskSetManager: TaskSet 407.0 using PreferredLocationsV1
25/02/06 06:11:38 INFO FairSchedulableBuilder: Added task set TaskSet_407.0 tasks
to pool 5400065285585087590
25/02/06 06:11:38 INFO DAGScheduler: Registering RDD 1414
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 226
25/02/06 06:11:38 INFO DAGScheduler: Got map stage job 229
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:38 INFO DAGScheduler: Final stage: ShuffleMapStage 410
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:38 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
409)
25/02/06 06:11:38 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:38 INFO DAGScheduler: Submitting ShuffleMapStage 410
(MapPartitionsRDD[1414] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:38 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 410 (MapPartitionsRDD[1414] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:38 INFO TaskSchedulerImpl: Adding task set 410.0 with 5 tasks
resource profile 0
25/02/06 06:11:38 INFO TaskSetManager: TaskSet 410.0 using PreferredLocationsV1
25/02/06 06:11:38 INFO FairSchedulableBuilder: Added task set TaskSet_410.0 tasks
to pool 5400065285585087590
25/02/06 06:11:38 INFO TaskSetManager: Starting task 1.0 in stage 200.0 (TID 916)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 8.0 in stage 196.0 (TID 905)
in 476 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.246.72:43231 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 59 to 10.183.246.72:39896
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:38 INFO TaskSetManager: Starting task 2.0 in stage 200.0 (TID 917)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 1.0 in stage 196.0 (TID 898)
in 909 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.232.45:45579 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 59 to 10.183.232.45:60256
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:38 INFO TaskSetManager: Starting task 3.0 in stage 200.0 (TID 918)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 0.0 in stage 196.0 (TID 897)
in 993 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.249.197:40023 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 59 to 10.183.249.197:36834
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:38 INFO TaskSetManager: Starting task 4.0 in stage 200.0 (TID 919)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:38 INFO TaskSetManager: Finished task 7.0 in stage 196.0 (TID 904)
in 738 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:38 INFO TaskSetManager: Finished task 8.0 in stage 198.0 (TID 914)
in 344 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:38 INFO TaskSetManager: Starting task 5.0 in stage 200.0 (TID 920)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:38 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on
10.183.233.191:42923 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 59 to 10.183.233.191:55538
25/02/06 06:11:38 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:38 INFO ShufflePartitionsUtil: For shuffle(137), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:39 INFO ShufflePartitionsUtil: For shuffle(138), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:39 INFO TaskSetManager: Starting task 6.0 in stage 200.0 (TID 921)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 4.0 in stage 196.0 (TID 901)
in 883 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 5.0 in stage 196.0 (TID 902)
in 903 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 7.0 in stage 200.0 (TID 922)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Starting task 8.0 in stage 200.0 (TID 923)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 6.0 in stage 196.0 (TID 903)
in 915 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 5.0 in stage 198.0 (TID 911)
in 737 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 0.0 in stage 202.0 (TID 924)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:39 INFO MemoryStore: Block broadcast_53 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:39 INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:39 INFO SparkContext: Created broadcast 53 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 61 to 10.183.249.197:36834
25/02/06 06:11:39 INFO TaskSetManager: Starting task 1.0 in stage 202.0 (TID 925)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 2.0 in stage 196.0 (TID 899)
in 1003 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 3.0 in stage 196.0 (TID 900)
in 1004 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:39 INFO TaskSchedulerImpl: Removed TaskSet 196.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:39 INFO TaskSetManager: Starting task 2.0 in stage 202.0 (TID 926)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:39 INFO DAGScheduler: ShuffleMapStage 196 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 105.173 s
25/02/06 06:11:39 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:39 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 198, ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238, ShuffleMapStage 311,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:39 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:39 INFO DAGScheduler: failed: Set()
25/02/06 06:11:39 INFO TaskSetManager: Starting task 3.0 in stage 202.0 (TID 927)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 4.0 in stage 198.0 (TID 910)
in 862 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 0.0 in stage 198.0 (TID 906)
in 984 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 4.0 in stage 202.0 (TID 928)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 5.0 in stage 202.0 (TID 929)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 1.0 in stage 198.0 (TID 907)
in 964 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 61 to 10.183.229.123:55848
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 61 to 10.183.232.45:60256
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 61 to 10.183.233.191:55538
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:39 INFO TaskSetManager: Starting task 6.0 in stage 202.0 (TID 930)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 3.0 in stage 198.0 (TID 909)
in 979 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 7.0 in stage 202.0 (TID 931)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 6.0 in stage 198.0 (TID 912)
in 878 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 2.0 in stage 198.0 (TID 908)
in 1117 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 8.0 in stage 202.0 (TID 932)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 8.0 in stage 200.0 (TID 923)
in 359 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 0.0 in stage 204.0 (TID 933)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:39 INFO MemoryStore: Block broadcast_54 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:39 INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:39 INFO SparkContext: Created broadcast 54 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 56 to 10.183.246.72:39896
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:39 INFO TaskSetManager: Starting task 1.0 in stage 204.0 (TID 934)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 7.0 in stage 198.0 (TID 913)
in 950 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:39 INFO TaskSchedulerImpl: Removed TaskSet 198.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:39 INFO DAGScheduler: ShuffleMapStage 198 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 103.394 s
25/02/06 06:11:39 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:39 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 200,
ShuffleMapStage 302, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273,
ShuffleMapStage 252, ShuffleMapStage 202, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:39 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:39 INFO DAGScheduler: failed: Set()
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 56 to 10.183.249.197:36834
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:39 INFO TaskSetManager: Starting task 2.0 in stage 204.0 (TID 935)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 3.0 in stage 200.0 (TID 918)
in 796 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:39 INFO TaskSetManager: Finished task 1.0 in stage 200.0 (TID 916)
in 922 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 3.0 in stage 204.0 (TID 936)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Starting task 4.0 in stage 204.0 (TID 937)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 8.0 in stage 202.0 (TID 932)
in 275 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 56 to 10.183.233.191:55538
25/02/06 06:11:39 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100279) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100279)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:39 INFO CodeGenerator: Code generated in 48.755914 ms
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:39 INFO TaskSetManager: Starting task 5.0 in stage 204.0 (TID 938)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 0.0 in stage 200.0 (TID 915)
in 1123 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:39 INFO CodeGenerator: Code generated in 42.309979 ms
25/02/06 06:11:39 INFO DAGScheduler: Registering RDD 1419
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 227
25/02/06 06:11:39 INFO DAGScheduler: Got map stage job 230
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:39 INFO DAGScheduler: Final stage: ShuffleMapStage 413
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:39 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
412)
25/02/06 06:11:39 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:39 INFO DAGScheduler: Submitting ShuffleMapStage 413
(MapPartitionsRDD[1419] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:39 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 413 (MapPartitionsRDD[1419] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:39 INFO TaskSchedulerImpl: Adding task set 413.0 with 6 tasks
resource profile 0
25/02/06 06:11:39 INFO TaskSetManager: TaskSet 413.0 using PreferredLocationsV1
25/02/06 06:11:39 INFO FairSchedulableBuilder: Added task set TaskSet_413.0 tasks
to pool 5400065285585087590
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:39 INFO DAGScheduler: Registering RDD 1424
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 228
25/02/06 06:11:39 INFO DAGScheduler: Got map stage job 231
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:39 INFO DAGScheduler: Final stage: ShuffleMapStage 416
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:39 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
415)
25/02/06 06:11:39 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:39 INFO DAGScheduler: Submitting ShuffleMapStage 416
(MapPartitionsRDD[1424] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 56 to 10.183.229.123:55848
25/02/06 06:11:39 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:39 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 416 (MapPartitionsRDD[1424] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:39 INFO TaskSchedulerImpl: Adding task set 416.0 with 6 tasks
resource profile 0
25/02/06 06:11:39 INFO TaskSetManager: TaskSet 416.0 using PreferredLocationsV1
25/02/06 06:11:39 INFO FairSchedulableBuilder: Added task set TaskSet_416.0 tasks
to pool 5400065285585087590
25/02/06 06:11:39 INFO TaskSetManager: Starting task 6.0 in stage 204.0 (TID 939)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 5.0 in stage 200.0 (TID 920)
in 860 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:39 INFO TaskSetManager: Starting task 7.0 in stage 204.0 (TID 940)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:39 INFO TaskSetManager: Finished task 2.0 in stage 200.0 (TID 917)
in 1114 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:39 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:40 INFO TaskSetManager: Starting task 8.0 in stage 204.0 (TID 941)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 0.0 in stage 206.0 (TID 942)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 1.0 in stage 206.0 (TID 943)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_55 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 2.0 in stage 206.0 (TID 944)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 56 to 10.183.232.45:60256
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:40 INFO SparkContext: Created broadcast 55 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:40 INFO TaskSetManager: Starting task 3.0 in stage 206.0 (TID 945)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:40 INFO BlockManagerInfo: Added taskresult_878 in memory on
10.183.232.45:45579 (size: 45.1 MiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 4.0 in stage 200.0 (TID 919)
in 1414 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 7.0 in stage 200.0 (TID 922)
in 1246 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 6.0 in stage 202.0 (TID 930)
in 1049 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 2.0 in stage 202.0 (TID 926)
in 1152 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 4.0 in stage 202.0 (TID 928)
in 1141 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 6.0 in stage 200.0 (TID 921)
in 1275 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:40 INFO TaskSchedulerImpl: Removed TaskSet 200.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:40 INFO DAGScheduler: ShuffleMapStage 200 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 102.571 s
25/02/06 06:11:40 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:40 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 302,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 202, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 246,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ResultStage 190, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:40 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:40 INFO DAGScheduler: failed: Set()
25/02/06 06:11:40 INFO TaskSetManager: Finished task 4.0 in stage 204.0 (TID 937)
in 677 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 1.0 in stage 202.0 (TID 925)
in 1157 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 3.0 in stage 204.0 (TID 936)
in 743 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 0.0 in stage 202.0 (TID 924)
in 1182 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 7.0 in stage 202.0 (TID 931)
in 974 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 3.0 in stage 202.0 (TID 927)
in 1148 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Finished task 5.0 in stage 202.0 (TID 929)
in 1147 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:40 INFO TaskSchedulerImpl: Removed TaskSet 202.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:40 INFO TaskSetManager: Finished task 0.0 in stage 204.0 (TID 933)
in 885 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 4.0 in stage 206.0 (TID 946)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 5.0 in stage 206.0 (TID 947)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 57 to 10.183.246.72:39896
25/02/06 06:11:40 INFO DAGScheduler: ShuffleMapStage 202 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 100.510 s
25/02/06 06:11:40 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:40 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 204, ShuffleMapStage 256,
ShuffleMapStage 206, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 302,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 248,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ResultStage 190,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 238,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 234, ShuffleMapStage 286,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 230,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 374,
ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:40 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:40 INFO DAGScheduler: failed: Set()
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:40 INFO TaskSetManager: Starting task 6.0 in stage 206.0 (TID 948)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 7.0 in stage 206.0 (TID 949)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 8.0 in stage 206.0 (TID 950)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:40 INFO TaskSetManager: Starting task 0.0 in stage 208.0 (TID 951)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 1.0 in stage 208.0 (TID 952)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 2.0 in stage 208.0 (TID 953)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_56 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 3.0 in stage 208.0 (TID 954)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 57 to 10.183.229.123:55848
25/02/06 06:11:40 INFO TaskSetManager: Finished task 1.0 in stage 204.0 (TID 934)
in 867 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 4.0 in stage 208.0 (TID 955)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Starting task 5.0 in stage 208.0 (TID 956)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:40 INFO SparkContext: Created broadcast 56 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:40 INFO TaskSetManager: Finished task 2.0 in stage 204.0 (TID 935)
in 779 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:40 INFO TaskSetManager: Starting task 6.0 in stage 208.0 (TID 957)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 57 to 10.183.233.191:55538
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:40 INFO TaskSetManager: Starting task 7.0 in stage 208.0 (TID 958)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Finished task 5.0 in stage 204.0 (TID 938)
in 663 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 55 to 10.183.249.197:36834
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 55 to 10.183.232.45:60256
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 55 to 10.183.229.123:55848
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:40 INFO TransportClientFactory: Successfully created connection
to /10.183.232.45:45579 after 24 ms (0 ms spent in bootstraps)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 8.0 in stage 208.0 (TID 959)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Finished task 6.0 in stage 204.0 (TID 939)
in 676 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:40 INFO ShufflePartitionsUtil: For shuffle(139), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:40 INFO ShufflePartitionsUtil: For shuffle(140), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:40 INFO TaskSetManager: Starting task 0.0 in stage 210.0 (TID 960)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Finished task 8.0 in stage 204.0 (TID 941)
in 400 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_57 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO MemoryStore: Block broadcast_57_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:40 INFO ShufflePartitionsUtil: For shuffle(141), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:40 INFO SparkContext: Created broadcast 57 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO ShufflePartitionsUtil: For shuffle(142), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 64 to 10.183.233.191:55538
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:40 INFO TaskSetManager: Finished task 0.0 in stage 190.0 (TID 878)
in 3801 ms on 10.183.232.45 (executor 2) (1/1)
25/02/06 06:11:40 INFO TaskSchedulerImpl: Removed TaskSet 190.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:40 INFO DAGScheduler: ResultStage 190
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
111.100 s
25/02/06 06:11:40 INFO DAGScheduler: Job 141 is finished. Cancelling potential
speculative or zombie tasks for this job
25/02/06 06:11:40 INFO TaskSchedulerImpl: Killing all running tasks in stage 190:
Stage finished
25/02/06 06:11:40 INFO DAGScheduler: Job 141 finished:
$anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63, took 111.106915
s
25/02/06 06:11:40 INFO BlockManagerInfo: Removed taskresult_878 on
10.183.232.45:45579 in memory (size: 45.1 MiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO TaskSetManager: Starting task 1.0 in stage 210.0 (TID 961)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Finished task 8.0 in stage 206.0 (TID 950)
in 527 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(214,1738822294098,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@220094f6, org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@770485c0,
org.apache.spark.scheduler.StageInfo@10833974),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 1.242138204s.
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 64 to 10.183.246.72:39896
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:40 INFO TaskSetManager: Starting task 2.0 in stage 210.0 (TID 962)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:40 INFO TaskSetManager: Finished task 8.0 in stage 208.0 (TID 959)
in 456 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:40 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 64 to 10.183.229.123:55848
25/02/06 06:11:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:41 INFO TaskSetManager: Starting task 3.0 in stage 210.0 (TID 963)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 2.0 in stage 206.0 (TID 944)
in 685 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 4.0 in stage 210.0 (TID 964)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 1.0 in stage 206.0 (TID 943)
in 807 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 4.0 in stage 208.0 (TID 955)
in 827 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 5.0 in stage 210.0 (TID 965)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 7.0 in stage 204.0 (TID 940)
in 1380 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:41 INFO TaskSchedulerImpl: Removed TaskSet 204.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: ShuffleMapStage 204 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 99.777 s
25/02/06 06:11:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:41 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 256, ShuffleMapStage 206,
ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410, ShuffleMapStage 279,
ShuffleMapStage 258, ShuffleMapStage 250, ShuffleMapStage 302, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401,
ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:41 INFO DAGScheduler: failed: Set()
25/02/06 06:11:41 INFO TaskSetManager: Starting task 6.0 in stage 210.0 (TID 966)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 64 to 10.183.249.197:36834
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 3.0 in stage 208.0 (TID 954)
in 860 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 7.0 in stage 210.0 (TID 967)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 64 to 10.183.232.45:60256
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:41 INFO TaskSetManager: Finished task 7.0 in stage 206.0 (TID 949)
in 902 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 8.0 in stage 210.0 (TID 968)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Starting task 0.0 in stage 212.0 (TID 969)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 6.0 in stage 206.0 (TID 948)
in 918 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:41 INFO MemoryStore: Block broadcast_58 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:41 INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:41 INFO SparkContext: Created broadcast 58 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 5.0 in stage 208.0 (TID 956)
in 935 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 1.0 in stage 212.0 (TID 970)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 65 to 10.183.246.72:39896
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:41 INFO TaskSetManager: Finished task 6.0 in stage 208.0 (TID 957)
in 939 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 2.0 in stage 212.0 (TID 971)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 65 to 10.183.232.45:60256
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 65 to 10.183.249.197:36834
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:41 INFO TaskSetManager: Starting task 3.0 in stage 212.0 (TID 972)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 4.0 in stage 206.0 (TID 946)
in 1008 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 0.0 in stage 206.0 (TID 942)
in 1039 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 4.0 in stage 212.0 (TID 973)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 65 to 10.183.229.123:55848
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:41 INFO TaskSetManager: Finished task 5.0 in stage 206.0 (TID 947)
in 1037 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 5.0 in stage 212.0 (TID 974)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 65 to 10.183.233.191:55538
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:41 INFO TaskSetManager: Starting task 6.0 in stage 212.0 (TID 975)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 3.0 in stage 206.0 (TID 945)
in 1076 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:41 INFO TaskSchedulerImpl: Removed TaskSet 206.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: ShuffleMapStage 206 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 96.916 s
25/02/06 06:11:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:41 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 256, ShuffleMapStage 329,
ShuffleMapStage 308, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 250, ShuffleMapStage 302, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 208, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:41 INFO DAGScheduler: failed: Set()
25/02/06 06:11:41 INFO TaskSetManager: Starting task 7.0 in stage 212.0 (TID 976)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 0.0 in stage 208.0 (TID 951)
in 1075 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 8.0 in stage 212.0 (TID 977)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:41 INFO TaskSetManager: Finished task 8.0 in stage 210.0 (TID 968)
in 267 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 1.0 in stage 208.0 (TID 952)
in 1181 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 0.0 in stage 214.0 (TID 978)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MemoryStore: Block broadcast_59 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:41 INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:41 INFO SparkContext: Created broadcast 59 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO TaskSetManager: Finished task 2.0 in stage 208.0 (TID 953)
in 1200 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 1.0 in stage 214.0 (TID 979)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 66 to 10.183.232.45:60256
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:41 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100477) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100477)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 66 to 10.183.249.197:36834
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:41 INFO TaskSetManager: Finished task 7.0 in stage 208.0 (TID 958)
in 1215 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:41 INFO TaskSchedulerImpl: Removed TaskSet 208.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: ShuffleMapStage 208 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 94.335 s
25/02/06 06:11:41 INFO TaskSetManager: Starting task 2.0 in stage 214.0 (TID 980)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:41 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 256, ShuffleMapStage 329,
ShuffleMapStage 308, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 250, ShuffleMapStage 302, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 234,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 228,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 222,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 218, ShuffleMapStage 320, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 210, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:41 INFO DAGScheduler: failed: Set()
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 66 to 10.183.229.123:55848
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:41 INFO CodeGenerator: Code generated in 75.406821 ms
25/02/06 06:11:41 INFO CodeGenerator: Code generated in 77.158146 ms
25/02/06 06:11:41 INFO DAGScheduler: Registering RDD 1432
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 229
25/02/06 06:11:41 INFO DAGScheduler: Got map stage job 232
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:11:41 INFO DAGScheduler: Final stage: ShuffleMapStage 419
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
418)
25/02/06 06:11:41 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:41 INFO DAGScheduler: Submitting ShuffleMapStage 419
(MapPartitionsRDD[1432] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:41 INFO CodeGenerator: Code generated in 75.724246 ms
25/02/06 06:11:41 INFO CodeGenerator: Code generated in 84.92317 ms
25/02/06 06:11:41 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 419 (MapPartitionsRDD[1432] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:11:41 INFO TaskSchedulerImpl: Adding task set 419.0 with 5 tasks
resource profile 0
25/02/06 06:11:41 INFO TaskSetManager: TaskSet 419.0 using PreferredLocationsV1
25/02/06 06:11:41 INFO FairSchedulableBuilder: Added task set TaskSet_419.0 tasks
to pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: Registering RDD 1442
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 231
25/02/06 06:11:41 INFO DAGScheduler: Got map stage job 233
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:41 INFO DAGScheduler: Final stage: ShuffleMapStage 422
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
421)
25/02/06 06:11:41 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:41 INFO DAGScheduler: Submitting ShuffleMapStage 422
(MapPartitionsRDD[1442] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:41 INFO DAGScheduler: Registering RDD 1444
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 232
25/02/06 06:11:41 INFO DAGScheduler: Got map stage job 234
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:41 INFO DAGScheduler: Final stage: ShuffleMapStage 425
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
424)
25/02/06 06:11:41 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:41 INFO DAGScheduler: Submitting ShuffleMapStage 425
(MapPartitionsRDD[1444] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:41 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 422 (MapPartitionsRDD[1442] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:11:41 INFO TaskSchedulerImpl: Adding task set 422.0 with 6 tasks
resource profile 0
25/02/06 06:11:41 INFO TaskSetManager: TaskSet 422.0 using PreferredLocationsV1
25/02/06 06:11:41 INFO FairSchedulableBuilder: Added task set TaskSet_422.0 tasks
to pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 425 (MapPartitionsRDD[1444] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:41 INFO TaskSchedulerImpl: Adding task set 425.0 with 6 tasks
resource profile 0
25/02/06 06:11:41 INFO TaskSetManager: TaskSet 425.0 using PreferredLocationsV1
25/02/06 06:11:41 INFO FairSchedulableBuilder: Added task set TaskSet_425.0 tasks
to pool 5400065285585087590
25/02/06 06:11:41 INFO DAGScheduler: Registering RDD 1440
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 230
25/02/06 06:11:41 INFO DAGScheduler: Got map stage job 235
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:11:41 INFO DAGScheduler: Final stage: ShuffleMapStage 428
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:11:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
427)
25/02/06 06:11:41 INFO DAGScheduler: Missing parents: List()
25/02/06 06:11:41 INFO DAGScheduler: Submitting ShuffleMapStage 428
(MapPartitionsRDD[1440] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:11:41 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 428 (MapPartitionsRDD[1440] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:11:41 INFO TaskSchedulerImpl: Adding task set 428.0 with 6 tasks
resource profile 0
25/02/06 06:11:41 INFO TaskSetManager: TaskSet 428.0 using PreferredLocationsV1
25/02/06 06:11:41 INFO FairSchedulableBuilder: Added task set TaskSet_428.0 tasks
to pool 5400065285585087590
25/02/06 06:11:41 INFO TaskSetManager: Finished task 0.0 in stage 210.0 (TID 960)
in 1011 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:41 INFO TaskSetManager: Starting task 3.0 in stage 214.0 (TID 981)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:41 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 66 to 10.183.233.191:55538
25/02/06 06:11:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:42 INFO TaskSetManager: Finished task 4.0 in stage 210.0 (TID 964)
in 1222 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 2.0 in stage 210.0 (TID 962)
in 1432 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 4.0 in stage 212.0 (TID 973)
in 987 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 5.0 in stage 212.0 (TID 974)
in 967 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 3.0 in stage 212.0 (TID 972)
in 1001 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 6.0 in stage 212.0 (TID 975)
in 945 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 2.0 in stage 212.0 (TID 971)
in 1098 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 6.0 in stage 210.0 (TID 966)
in 1202 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 7.0 in stage 212.0 (TID 976)
in 977 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 0.0 in stage 214.0 (TID 978)
in 870 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 5.0 in stage 210.0 (TID 965)
in 1219 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 2.0 in stage 214.0 (TID 980)
in 809 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 1.0 in stage 212.0 (TID 970)
in 1111 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 7.0 in stage 210.0 (TID 967)
in 1190 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 1.0 in stage 214.0 (TID 979)
in 852 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 1.0 in stage 210.0 (TID 961)
in 1530 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 8.0 in stage 212.0 (TID 977)
in 886 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:42 INFO TaskSetManager: Starting task 4.0 in stage 214.0 (TID 982)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 5.0 in stage 214.0 (TID 983)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 6.0 in stage 214.0 (TID 984)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 7.0 in stage 214.0 (TID 985)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 8.0 in stage 214.0 (TID 986)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 0.0 in stage 216.0 (TID 987)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 1.0 in stage 216.0 (TID 988)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 2.0 in stage 216.0 (TID 989)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 3.0 in stage 216.0 (TID 990)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 4.0 in stage 216.0 (TID 991)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 5.0 in stage 216.0 (TID 992)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 6.0 in stage 216.0 (TID 993)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 7.0 in stage 216.0 (TID 994)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 8.0 in stage 216.0 (TID 995)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 0.0 in stage 218.0 (TID 996)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 1.0 in stage 218.0 (TID 997)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:42 INFO MemoryStore: Block broadcast_60 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:42 INFO TaskSetManager: Starting task 2.0 in stage 218.0 (TID 998)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 3.0 in stage 218.0 (TID 999)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 4.0 in stage 218.0 (TID 1000)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Starting task 5.0 in stage 218.0 (TID 1001)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Finished task 3.0 in stage 210.0 (TID 963)
in 1416 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:42 INFO TaskSchedulerImpl: Removed TaskSet 210.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:42 INFO DAGScheduler: ShuffleMapStage 210 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 93.928 s
25/02/06 06:11:42 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:42 INFO DAGScheduler: running: Set(ShuffleMapStage 212,
ShuffleMapStage 335, ShuffleMapStage 356, ShuffleMapStage 256, ShuffleMapStage 329,
ShuffleMapStage 308, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258,
ShuffleMapStage 250, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 242, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240,
ShuffleMapStage 232, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 220,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 214, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 216, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:42 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:42 INFO DAGScheduler: failed: Set()
25/02/06 06:11:42 INFO TaskSetManager: Finished task 0.0 in stage 212.0 (TID 969)
in 1167 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:42 INFO TaskSchedulerImpl: Removed TaskSet 212.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:42 INFO TaskSetManager: Finished task 3.0 in stage 214.0 (TID 981)
in 703 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:42 INFO DAGScheduler: ShuffleMapStage 212 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 91.271 s
25/02/06 06:11:42 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:42 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 216,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:42 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:42 INFO DAGScheduler: failed: Set()
25/02/06 06:11:42 INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:42 INFO SparkContext: Created broadcast 60 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:42 INFO MemoryStore: Block broadcast_61 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:42 INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:42 INFO SparkContext: Created broadcast 61 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 62 to 10.183.232.45:60256
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 62 to 10.183.229.123:55848
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 63 to 10.183.246.72:39896
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 62 to 10.183.249.197:36834
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 63 to 10.183.249.197:36834
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:42 INFO TaskSetManager: Starting task 6.0 in stage 218.0 (TID 1002)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:42 INFO TaskSetManager: Finished task 8.0 in stage 216.0 (TID 995)
in 502 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:42 INFO TaskSetManager: Finished task 8.0 in stage 214.0 (TID 986)
in 506 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:42 INFO TaskSetManager: Starting task 7.0 in stage 218.0 (TID 1003)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:42 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 63 to 10.183.229.123:55848
25/02/06 06:11:42 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:43 INFO TaskSetManager: Starting task 8.0 in stage 218.0 (TID 1004)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 7.0 in stage 216.0 (TID 994)
in 710 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 0.0 in stage 220.0 (TID 1005)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 2.0 in stage 218.0 (TID 998)
in 764 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:43 INFO MemoryStore: Block broadcast_62 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:43 INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:43 INFO SparkContext: Created broadcast 62 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 1.0 in stage 220.0 (TID 1006)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 6.0 in stage 214.0 (TID 984)
in 812 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 67 to 10.183.246.72:39896
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 67 to 10.183.233.191:55538
25/02/06 06:11:43 INFO TaskSetManager: Starting task 2.0 in stage 220.0 (TID 1007)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 4.0 in stage 214.0 (TID 982)
in 830 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:43 INFO TaskSetManager: Finished task 3.0 in stage 216.0 (TID 990)
in 841 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 3.0 in stage 220.0 (TID 1008)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Starting task 4.0 in stage 220.0 (TID 1009)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 4.0 in stage 216.0 (TID 991)
in 846 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 67 to 10.183.232.45:60256
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:43 INFO TaskSetManager: Starting task 5.0 in stage 220.0 (TID 1010)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 4.0 in stage 218.0 (TID 1000)
in 895 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 6.0 in stage 220.0 (TID 1011)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 6.0 in stage 216.0 (TID 993)
in 961 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:43 INFO TaskSetManager: Finished task 1.0 in stage 216.0 (TID 988)
in 966 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 7.0 in stage 220.0 (TID 1012)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 5.0 in stage 218.0 (TID 1001)
in 962 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 8.0 in stage 220.0 (TID 1013)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 0.0 in stage 222.0 (TID 1014)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 1.0 in stage 218.0 (TID 997)
in 986 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:43 INFO MemoryStore: Block broadcast_63 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:43 INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:43 INFO SparkContext: Created broadcast 63 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 68 to 10.183.249.197:36834
25/02/06 06:11:43 INFO TaskSetManager: Starting task 1.0 in stage 222.0 (TID 1015)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 2.0 in stage 216.0 (TID 989)
in 1008 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 67 to 10.183.229.123:55848
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 2.0 in stage 222.0 (TID 1016)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 3.0 in stage 218.0 (TID 999)
in 1030 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:43 INFO TaskSetManager: Finished task 0.0 in stage 216.0 (TID 987)
in 1042 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 3.0 in stage 222.0 (TID 1017)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 68 to 10.183.229.123:55848
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 68 to 10.183.246.72:39896
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:43 INFO TaskSetManager: Starting task 4.0 in stage 222.0 (TID 1018)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:43 INFO TaskSetManager: Finished task 5.0 in stage 214.0 (TID 983)
in 1108 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:11:43 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:43 INFO TaskSetManager: Finished task 8.0 in stage 218.0 (TID 1004)
in 412 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:11:43 INFO TaskSetManager: Starting task 5.0 in stage 222.0 (TID 1019)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 68 to 10.183.233.191:55538
25/02/06 06:11:43 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:45 INFO TaskSetManager: Finished task 6.0 in stage 218.0 (TID 1002)
in 2142 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 0.0 in stage 218.0 (TID 996)
in 2645 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 0.0 in stage 222.0 (TID 1014)
in 1660 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 5.0 in stage 216.0 (TID 992)
in 2648 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:45 INFO TaskSchedulerImpl: Removed TaskSet 216.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:45 INFO TaskSetManager: Finished task 6.0 in stage 220.0 (TID 1011)
in 1694 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 4.0 in stage 220.0 (TID 1009)
in 1809 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:11:45 INFO DAGScheduler: ShuffleMapStage 216 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 89.757 s
25/02/06 06:11:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 214,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:45 INFO DAGScheduler: failed: Set()
25/02/06 06:11:45 INFO TaskSetManager: Finished task 7.0 in stage 214.0 (TID 985)
in 2664 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:11:45 INFO TaskSchedulerImpl: Removed TaskSet 214.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:45 INFO TaskSetManager: Finished task 2.0 in stage 220.0 (TID 1007)
in 1835 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:45 INFO DAGScheduler: ShuffleMapStage 214 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 90.678 s
25/02/06 06:11:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:45 INFO TaskSetManager: Starting task 6.0 in stage 222.0 (TID 1020)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 7.0 in stage 222.0 (TID 1021)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 8.0 in stage 222.0 (TID 1022)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 0.0 in stage 224.0 (TID 1023)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_64 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:45 INFO SparkContext: Created broadcast 64 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 218, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:45 INFO DAGScheduler: failed: Set()
25/02/06 06:11:45 INFO TaskSetManager: Starting task 1.0 in stage 224.0 (TID 1024)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 2.0 in stage 224.0 (TID 1025)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 3.0 in stage 224.0 (TID 1026)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 4.0 in stage 224.0 (TID 1027)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 5.0 in stage 224.0 (TID 1028)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 6.0 in stage 224.0 (TID 1029)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 7.0 in stage 224.0 (TID 1030)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 8.0 in stage 224.0 (TID 1031)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 4.0 in stage 222.0 (TID 1018)
in 1590 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 7.0 in stage 218.0 (TID 1003)
in 2193 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:45 INFO TaskSchedulerImpl: Removed TaskSet 218.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:45 INFO TaskSetManager: Finished task 5.0 in stage 222.0 (TID 1019)
in 1574 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 3.0 in stage 222.0 (TID 1017)
in 1655 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 0.0 in stage 226.0 (TID 1032)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 1.0 in stage 226.0 (TID 1033)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 2.0 in stage 226.0 (TID 1034)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 3.0 in stage 226.0 (TID 1035)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:45 INFO DAGScheduler: ShuffleMapStage 218 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 87.185 s
25/02/06 06:11:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_65 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 220, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:45 INFO DAGScheduler: failed: Set()
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 72 to 10.183.249.197:36834
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:45 INFO SparkContext: Created broadcast 65 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 72 to 10.183.233.191:55538
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:45 INFO TaskSetManager: Starting task 4.0 in stage 226.0 (TID 1036)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 5.0 in stage 226.0 (TID 1037)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 1.0 in stage 220.0 (TID 1006)
in 1906 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 8.0 in stage 220.0 (TID 1013)
in 1744 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 3.0 in stage 220.0 (TID 1008)
in 1877 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 5.0 in stage 220.0 (TID 1010)
in 1819 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 71 to 10.183.229.123:55848
25/02/06 06:11:45 INFO TaskSetManager: Finished task 0.0 in stage 220.0 (TID 1005)
in 1950 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 1.0 in stage 222.0 (TID 1015)
in 1717 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 7.0 in stage 220.0 (TID 1012)
in 1759 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:11:45 INFO TaskSchedulerImpl: Removed TaskSet 220.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:45 INFO TaskSetManager: Finished task 2.0 in stage 222.0 (TID 1016)
in 1689 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 72 to 10.183.232.45:60256
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 71 to 10.183.246.72:39896
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:45 INFO DAGScheduler: ShuffleMapStage 220 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 85.642 s
25/02/06 06:11:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 222, ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 226, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:45 INFO DAGScheduler: failed: Set()
25/02/06 06:11:45 INFO TaskSetManager: Starting task 6.0 in stage 226.0 (TID 1038)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 7.0 in stage 226.0 (TID 1039)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 8.0 in stage 226.0 (TID 1040)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 8.0 in stage 224.0 (TID 1031)
in 321 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 0.0 in stage 228.0 (TID 1041)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 8.0 in stage 222.0 (TID 1022)
in 458 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_66 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:11:45 INFO SparkContext: Created broadcast 66 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 71 to 10.183.233.191:55538
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 69 to 10.183.249.197:36834
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:45 INFO TaskSetManager: Finished task 4.0 in stage 226.0 (TID 1036)
in 659 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 1.0 in stage 228.0 (TID 1042)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 8.0 in stage 226.0 (TID 1040)
in 367 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 2.0 in stage 228.0 (TID 1043)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 7.0 in stage 222.0 (TID 1021)
in 717 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 3.0 in stage 228.0 (TID 1044)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 69 to 10.183.233.191:55538
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 69 to 10.183.246.72:39896
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:45 INFO TaskSetManager: Starting task 4.0 in stage 228.0 (TID 1045)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 6.0 in stage 224.0 (TID 1029)
in 757 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:11:45 INFO TaskSetManager: Finished task 5.0 in stage 224.0 (TID 1028)
in 757 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 5.0 in stage 228.0 (TID 1046)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Finished task 5.0 in stage 226.0 (TID 1037)
in 737 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:11:45 INFO TaskSetManager: Starting task 6.0 in stage 228.0 (TID 1047)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 7.0 in stage 228.0 (TID 1048)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:45 INFO TaskSetManager: Starting task 8.0 in stage 228.0 (TID 1049)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 3.0 in stage 226.0 (TID 1035)
in 927 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:11:46 INFO TaskSetManager: Finished task 1.0 in stage 224.0 (TID 1024)
in 944 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO TaskSetManager: Finished task 6.0 in stage 222.0 (TID 1020)
in 969 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:11:46 INFO TaskSchedulerImpl: Removed TaskSet 222.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:46 INFO TaskSetManager: Finished task 3.0 in stage 224.0 (TID 1026)
in 951 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:46 INFO DAGScheduler: ShuffleMapStage 222 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 84.361 s
25/02/06 06:11:46 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:46 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 374, ShuffleMapStage 224, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 226, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:46 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:46 INFO DAGScheduler: failed: Set()
25/02/06 06:11:46 INFO TaskSetManager: Starting task 0.0 in stage 230.0 (TID 1050)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Starting task 1.0 in stage 230.0 (TID 1051)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:11:46 INFO MemoryStore: Block broadcast_67 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:46 INFO SparkContext: Created broadcast 67 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 69 to 10.183.232.45:60256
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO TaskSetManager: Starting task 2.0 in stage 230.0 (TID 1052)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 1.0 in stage 226.0 (TID 1033)
in 950 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:11:46 INFO TaskSetManager: Starting task 3.0 in stage 230.0 (TID 1053)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 0.0 in stage 226.0 (TID 1032)
in 956 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 69 to 10.183.229.123:55848
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 70 to 10.183.232.45:60256
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 70 to 10.183.249.197:36834
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 70 to 10.183.229.123:55848
25/02/06 06:11:46 INFO TaskSetManager: Starting task 4.0 in stage 230.0 (TID 1054)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 7.0 in stage 224.0 (TID 1030)
in 1009 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO TaskSetManager: Finished task 0.0 in stage 224.0 (TID 1023)
in 1043 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:11:46 INFO TaskSetManager: Starting task 5.0 in stage 230.0 (TID 1055)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 70 to 10.183.233.191:55538
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:46 INFO TaskSetManager: Starting task 6.0 in stage 230.0 (TID 1056)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 4.0 in stage 224.0 (TID 1027)
in 1038 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:46 INFO TaskSetManager: Finished task 2.0 in stage 224.0 (TID 1025)
in 1116 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:46 INFO TaskSchedulerImpl: Removed TaskSet 224.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:46 INFO DAGScheduler: ShuffleMapStage 224 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 81.300 s
25/02/06 06:11:46 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:46 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 374, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 226,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:46 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:46 INFO DAGScheduler: failed: Set()
25/02/06 06:11:46 INFO TaskSetManager: Starting task 7.0 in stage 230.0 (TID 1057)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Starting task 8.0 in stage 230.0 (TID 1058)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 7.0 in stage 226.0 (TID 1039)
in 1056 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 70 to 10.183.246.72:39896
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:46 INFO TaskSetManager: Starting task 0.0 in stage 232.0 (TID 1059)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 2.0 in stage 226.0 (TID 1034)
in 1187 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:11:46 INFO MemoryStore: Block broadcast_68 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:46 INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:46 INFO SparkContext: Created broadcast 68 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 51 to 10.183.229.123:55848
25/02/06 06:11:46 INFO TaskSetManager: Starting task 1.0 in stage 232.0 (TID 1060)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 0.0 in stage 228.0 (TID 1041)
in 810 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 51 to 10.183.249.197:36834
25/02/06 06:11:46 INFO TaskSetManager: Finished task 8.0 in stage 228.0 (TID 1049)
in 446 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:46 INFO TaskSetManager: Starting task 2.0 in stage 232.0 (TID 1061)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:11:46 INFO TaskSetManager: Finished task 6.0 in stage 226.0 (TID 1038)
in 1220 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:46 INFO TaskSchedulerImpl: Removed TaskSet 226.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:46 INFO DAGScheduler: ShuffleMapStage 226 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 80.544 s
25/02/06 06:11:46 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:46 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 228, ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 374, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:46 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:46 INFO DAGScheduler: failed: Set()
25/02/06 06:11:46 INFO TaskSetManager: Starting task 3.0 in stage 232.0 (TID 1062)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:46 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:46 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 51 to 10.183.246.72:39896
25/02/06 06:11:48 INFO TaskSetManager: Finished task 2.0 in stage 230.0 (TID 1052)
in 2243 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:48 INFO TaskSetManager: Starting task 4.0 in stage 232.0 (TID 1063)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Finished task 0.0 in stage 232.0 (TID 1059)
in 2009 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 3.0 in stage 228.0 (TID 1044)
in 2510 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 3.0 in stage 230.0 (TID 1053)
in 2244 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 0.0 in stage 230.0 (TID 1050)
in 2261 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 5.0 in stage 230.0 (TID 1055)
in 2188 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 1.0 in stage 232.0 (TID 1060)
in 1967 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 7.0 in stage 228.0 (TID 1048)
in 2378 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 1.0 in stage 230.0 (TID 1051)
in 2262 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 2.0 in stage 232.0 (TID 1061)
in 1933 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 2.0 in stage 228.0 (TID 1043)
in 2525 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 4.0 in stage 228.0 (TID 1045)
in 2461 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 5.0 in stage 228.0 (TID 1046)
in 2461 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 4.0 in stage 230.0 (TID 1054)
in 2210 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 8.0 in stage 230.0 (TID 1058)
in 2098 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 1.0 in stage 228.0 (TID 1042)
in 2535 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 6.0 in stage 228.0 (TID 1047)
in 2458 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:11:48 INFO TaskSchedulerImpl: Removed TaskSet 228.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:48 INFO DAGScheduler: ShuffleMapStage 228 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 79.692 s
25/02/06 06:11:48 INFO TaskSetManager: Finished task 3.0 in stage 232.0 (TID 1062)
in 1935 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:11:48 INFO ShufflePartitionsUtil: For shuffle(145), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 230, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 374,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:11:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:48 INFO DAGScheduler: failed: Set()
25/02/06 06:11:48 INFO TaskSetManager: Starting task 5.0 in stage 232.0 (TID 1064)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 6.0 in stage 232.0 (TID 1065)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 7.0 in stage 232.0 (TID 1066)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 8.0 in stage 232.0 (TID 1067)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 0.0 in stage 234.0 (TID 1068)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 1.0 in stage 234.0 (TID 1069)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 2.0 in stage 234.0 (TID 1070)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 3.0 in stage 234.0 (TID 1071)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:11:48 INFO MemoryStore: Block broadcast_69 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:48 INFO TaskSetManager: Starting task 4.0 in stage 234.0 (TID 1072)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 5.0 in stage 234.0 (TID 1073)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 6.0 in stage 234.0 (TID 1074)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:11:48 INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:48 INFO SparkContext: Created broadcast 69 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:48 INFO TaskSetManager: Starting task 7.0 in stage 234.0 (TID 1075)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 8.0 in stage 234.0 (TID 1076)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 0.0 in stage 236.0 (TID 1077)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 1.0 in stage 236.0 (TID 1078)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 2.0 in stage 236.0 (TID 1079)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 3.0 in stage 236.0 (TID 1080)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 4.0 in stage 236.0 (TID 1081)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Starting task 5.0 in stage 236.0 (TID 1082)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:11:48 INFO MemoryStore: Block broadcast_70 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 6.0 in stage 230.0 (TID 1056)
in 2267 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:11:48 INFO TaskSetManager: Finished task 7.0 in stage 230.0 (TID 1057)
in 2185 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:11:48 INFO TaskSchedulerImpl: Removed TaskSet 230.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:11:48 INFO DAGScheduler: ShuffleMapStage 230 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 77.043 s
25/02/06 06:11:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:11:48 INFO MemoryStore: Block broadcast_70_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 234, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:11:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:11:48 INFO DAGScheduler: failed: Set()
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:48 INFO SparkContext: Created broadcast 70 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 73 to 10.183.246.72:39896
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 74 to 10.183.246.72:39896
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 74 to 10.183.233.191:55538
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 73 to 10.183.232.45:60256
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 73 to 10.183.249.197:36834
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 73 to 10.183.233.191:55538
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:48 INFO TaskSetManager: Finished task 8.0 in stage 232.0 (TID 1067)
in 257 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:11:48 INFO TaskSetManager: Starting task 6.0 in stage 236.0 (TID 1083)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:11:48 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 74 to 10.183.249.197:36834
25/02/06 06:11:48 INFO ShufflePartitionsUtil: For shuffle(146), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:48 INFO TaskSetManager: Starting task 7.0 in stage 236.0 (TID 1084)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:11:48 INFO TaskSetManager: Finished task 8.0 in stage 234.0 (TID 1076)
in 380 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:11:48 INFO ShufflePartitionsUtil: For shuffle(144), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:49 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:11:49 INFO ShufflePartitionsUtil: For shuffle(143), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:11:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:11:49 INFO TaskSetManager: Starting task 8.0 in stage 236.0 (TID 1085)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:11:49 INFO TaskSetManager: Finished task 7.0 in stage 232.0 (TID 1066)
in 760 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:11:49 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 74 to 10.183.229.123:55848
25/02/06 06:11:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:11:49 INFO TaskSetManager: Starting task 0.0 in stage 238.0 (TID 1086)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:11:49 INFO TaskSetManager: Finished task 2.0 in stage 234.0 (TID 1070)
in 851 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:11:49 INFO MemoryStore: Block broadcast_71 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:11:49 INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:11:49 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:11:49 INFO SparkContext: Created broadcast 71 from broadcast at
TaskSetManager.scala:711
25/02/06 06:11:49 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 76 to 10.183.249.197:36834
25/02/06 06:11:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:11:49 INFO TaskSetManager: Finished task 4.0 in stage 232.0 (TID 1063)
in 958 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:11:49 INFO TaskSetManager: Starting task 1.0 in stage 238.0 (TID 1087)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:11:49 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:11:49 INFO TaskSetManager: Finished task 6.0 in stage 234.0 (TID 1074)
in 902 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:11:49 INFO TaskSetManager: Starting task 2.0 in stage 238.0 (TID 1088)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:11:49 INFO TaskSetManager: Finished task 4.0 in stage 236.0 (TID 1081)
in 883 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:11:49 INFO TaskSetManager: Starting task 3.0 in stage 238.0 (TID 1089)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:11:49 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:16 WARN HikariPool: HikariPool-1 - Thread starvation or clock leap
detected (housekeeper delta=57s275ms69µs612ns).
25/02/06 06:12:16 WARN HikariPool: HikariPool-2 - Thread starvation or clock leap
detected (housekeeper delta=57s112ms780µs530ns).
25/02/06 06:12:16 INFO TaskSetManager: Finished task 1.0 in stage 236.0 (TID 1078)
in 28522 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 3.0 in stage 236.0 (TID 1080)
in 28521 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 5.0 in stage 236.0 (TID 1082)
in 28522 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 7.0 in stage 236.0 (TID 1084)
in 28144 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 4.0 in stage 234.0 (TID 1072)
in 28558 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 5.0 in stage 234.0 (TID 1073)
in 28558 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 1.0 in stage 234.0 (TID 1069)
in 28574 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 0.0 in stage 234.0 (TID 1068)
in 28574 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 6.0 in stage 236.0 (TID 1083)
in 28317 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 0.0 in stage 238.0 (TID 1086)
in 27726 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 3.0 in stage 234.0 (TID 1071)
in 28568 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:12:16 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:16 INFO TaskSetManager: Finished task 0.0 in stage 236.0 (TID 1077)
in 28553 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:12:16 INFO TaskSetManager: Finished task 7.0 in stage 234.0 (TID 1075)
in 28555 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:12:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 76 to 10.183.229.123:55848
25/02/06 06:12:16 INFO TaskSchedulerImpl: Removed TaskSet 234.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO TaskSetManager: Starting task 4.0 in stage 238.0 (TID 1090)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 5.0 in stage 238.0 (TID 1091)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 6.0 in stage 238.0 (TID 1092)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 7.0 in stage 238.0 (TID 1093)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 8.0 in stage 238.0 (TID 1094)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 0.0 in stage 240.0 (TID 1095)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 1.0 in stage 240.0 (TID 1096)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 2.0 in stage 240.0 (TID 1097)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 3.0 in stage 240.0 (TID 1098)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 4.0 in stage 240.0 (TID 1099)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 5.0 in stage 240.0 (TID 1100)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 6.0 in stage 240.0 (TID 1101)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 7.0 in stage 240.0 (TID 1102)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 8.0 in stage 240.0 (TID 1103)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 0.0 in stage 242.0 (TID 1104)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 1.0 in stage 242.0 (TID 1105)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:12:17 INFO TaskSetManager: Starting task 2.0 in stage 242.0 (TID 1106)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:12:17 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO MemoryStore: Block broadcast_73 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:17 INFO DAGScheduler: ShuffleMapStage 234 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 101.552 s
25/02/06 06:12:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 76 to 10.183.232.45:60256
25/02/06 06:12:17 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 236, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:17 INFO DAGScheduler: failed: Set()
25/02/06 06:12:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_19_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_19_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_18_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_6_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO MemoryStore: Block broadcast_72 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:17 INFO MemoryStore: Block broadcast_72_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:17 INFO SparkContext: Created broadcast 72 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:17 INFO SparkContext: Created broadcast 73 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 76 to 10.183.246.72:39896
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_18_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_19_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_18_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_6_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.232.45:45579 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_59_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_18_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_19_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO TaskSetManager: Finished task 2.0 in stage 236.0 (TID 1079)
in 29103 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO TaskSetManager: Finished task 8.0 in stage 236.0 (TID 1085)
in 28401 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:12:17 INFO TaskSchedulerImpl: Removed TaskSet 236.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:17 INFO TaskSetManager: Finished task 5.0 in stage 232.0 (TID 1064)
in 29159 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:12:17 INFO TaskSetManager: Finished task 6.0 in stage 232.0 (TID 1065)
in 29159 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:12:17 INFO TaskSchedulerImpl: Removed TaskSet 232.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:17 INFO DAGScheduler: ShuffleMapStage 236 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 100.096 s
25/02/06 06:12:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:17 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 232, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:17 INFO DAGScheduler: failed: Set()
25/02/06 06:12:17 INFO DAGScheduler: ShuffleMapStage 232 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 105.247 s
25/02/06 06:12:17 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:17 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 238, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 240, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:17 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:17 INFO DAGScheduler: failed: Set()
25/02/06 06:12:17 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(216,1738822294138,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@34f76ab1, org.apache.spark.scheduler.StageInfo@1276effd,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@410f42f6),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 36.025781062s.
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.249.197:40023 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_22_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_44_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_22_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO BlockManagerInfo: Removed broadcast_45_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:17 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:12:17 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:18 INFO TaskSetManager: Starting task 3.0 in stage 242.0 (TID 1107)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:12:18 INFO TaskSetManager: Starting task 4.0 in stage 242.0 (TID 1108)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:12:18 INFO TaskSetManager: Finished task 5.0 in stage 238.0 (TID 1091)
in 1007 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:12:18 INFO TaskSetManager: Finished task 1.0 in stage 238.0 (TID 1087)
in 28764 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:12:18 INFO TaskSetManager: Finished task 2.0 in stage 238.0 (TID 1088)
in 28758 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:12:18 INFO TaskSetManager: Starting task 5.0 in stage 242.0 (TID 1109)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:12:18 INFO TaskSetManager: Starting task 6.0 in stage 242.0 (TID 1110)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:12:18 INFO TaskSetManager: Finished task 6.0 in stage 238.0 (TID 1092)
in 1015 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 76 to 10.183.233.191:55538
25/02/06 06:12:18 INFO BlockManagerInfo: Removed broadcast_28_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_21_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:28 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:12:28 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_19_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10982.052771 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11054.525976 ms
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_6_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_59_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_3_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_56_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_13_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO TaskSetManager: Starting task 7.0 in stage 242.0 (TID 1111)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:12:28 INFO TaskSetManager: Finished task 7.0 in stage 238.0 (TID 1093)
in 11568 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10983.501972 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10952.526209 ms
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_18_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11043.788878 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10976.655055 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10988.381428 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10985.22581 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11080.741971 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11181.033852 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11201.241809 ms
25/02/06 06:12:28 INFO AsyncEventQueue: Process of event
SparkListenerSQLAdaptiveExecutionUpdate(executionId=32, ...) by listener
SQLAppStatusListener took 10.880684313s.
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11078.055261 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11101.629085 ms
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO TaskSetManager: Starting task 8.0 in stage 242.0 (TID 1112)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:12:28 INFO TaskSetManager: Finished task 8.0 in stage 238.0 (TID 1094)
in 11738 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11188.190706 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11123.234556 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11279.2718 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11137.669993 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 10854.713812 ms
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_8_piece0 on
10.183.246.72:43231 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_22_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_44_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.246.72:43231 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_11_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_25_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_13_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_34_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_22_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_44_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO BlockManagerInfo: Removed broadcast_39_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11324.651393 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11202.734082 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11312.344578 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11411.683235 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11277.328051 ms
25/02/06 06:12:28 INFO CodeGenerator: Code generated in 11423.454666 ms
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#167080), DISTR_CHAN#167074 IN (20,25),
COMP_CODE#167072 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#167122),
isnotnull(QUANT_B#167109), (BILL_DATE#167122 = 2022-11-26),
isnotnull(BILL_TYPE#167123), NOT (QUANT_B#167109 = 0.000), NOT (BILL_TYPE#167123 =
ZF8), NOT (BILL_TYPE#167123 = ZS1), NOT (BILL_TYPE#167123 = F8), NOT
(BILL_TYPE#167123 = ZP02), dynamicpruning#308712 308711)
Actual: Set(isnotnull(BILL_DATE#167122), isnotnull(QUANT_B#167109),
(BILL_DATE#167122 = 2022-11-26), isnotnull(BILL_TYPE#167123),
isnotnull(MATERIAL#167080), NOT (QUANT_B#167109 = 0.000), NOT (BILL_TYPE#167123 =
ZF8), NOT (BILL_TYPE#167123 = ZS1), NOT (BILL_TYPE#167123 = F8), NOT
(BILL_TYPE#167123 = ZP02), DISTR_CHAN#167074 IN (20,25), COMP_CODE#167072 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#167080 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#102880), DISTR_CHAN#102874 IN (20,25),
COMP_CODE#102872 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#102922),
isnotnull(QUANT_B#102909), (BILL_DATE#102922 = 2022-11-20),
isnotnull(BILL_TYPE#102923), NOT (QUANT_B#102909 = 0.000), NOT (BILL_TYPE#102923 =
ZF8), NOT (BILL_TYPE#102923 = ZS1), NOT (BILL_TYPE#102923 = F8), NOT
(BILL_TYPE#102923 = ZP02), dynamicpruning#308700 308699)
Actual: Set(isnotnull(BILL_DATE#102922), isnotnull(QUANT_B#102909),
(BILL_DATE#102922 = 2022-11-20), isnotnull(BILL_TYPE#102923),
isnotnull(MATERIAL#102880), NOT (QUANT_B#102909 = 0.000), NOT (BILL_TYPE#102923 =
ZF8), NOT (BILL_TYPE#102923 = ZS1), NOT (BILL_TYPE#102923 = F8), NOT
(BILL_TYPE#102923 = ZP02), DISTR_CHAN#102874 IN (20,25), COMP_CODE#102872 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#102880 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#1432), DISTR_CHAN#1426 IN (20,25), COMP_CODE#1424
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#1474),
isnotnull(QUANT_B#1461), (BILL_DATE#1474 = 2022-11-02), isnotnull(BILL_TYPE#1475),
NOT (QUANT_B#1461 = 0.000), NOT (BILL_TYPE#1475 = ZF8), NOT (BILL_TYPE#1475 = ZS1),
NOT (BILL_TYPE#1475 = F8), NOT (BILL_TYPE#1475 = ZP02), dynamicpruning#308664
308663)
Actual: Set(isnotnull(BILL_DATE#1474), isnotnull(QUANT_B#1461), (BILL_DATE#1474 =
2022-11-02), isnotnull(BILL_TYPE#1475), isnotnull(MATERIAL#1432), NOT (QUANT_B#1461
= 0.000), NOT (BILL_TYPE#1475 = ZF8), NOT (BILL_TYPE#1475 = ZS1), NOT
(BILL_TYPE#1475 = F8), NOT (BILL_TYPE#1475 = ZP02), DISTR_CHAN#1426 IN (20,25),
COMP_CODE#1424 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#1432 IN
dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#122592), DISTR_CHAN#122586 IN (20,25),
COMP_CODE#122584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#122634),
isnotnull(QUANT_B#122621), (BILL_DATE#122634 = 2022-11-22),
isnotnull(BILL_TYPE#122635), NOT (QUANT_B#122621 = 0.000), NOT (BILL_TYPE#122635 =
ZF8), NOT (BILL_TYPE#122635 = ZS1), NOT (BILL_TYPE#122635 = F8), NOT
(BILL_TYPE#122635 = ZP02), dynamicpruning#308704 308703)
Actual: Set(isnotnull(BILL_DATE#122634), isnotnull(QUANT_B#122621),
(BILL_DATE#122634 = 2022-11-22), isnotnull(BILL_TYPE#122635),
isnotnull(MATERIAL#122592), NOT (QUANT_B#122621 = 0.000), NOT (BILL_TYPE#122635 =
ZF8), NOT (BILL_TYPE#122635 = ZS1), NOT (BILL_TYPE#122635 = F8), NOT
(BILL_TYPE#122635 = ZP02), DISTR_CHAN#122586 IN (20,25), COMP_CODE#122584 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#122592 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#370), DISTR_CHAN#364 IN (20,25), COMP_CODE#362
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#412),
isnotnull(QUANT_B#399), (BILL_DATE#412 = 2022-11-01), isnotnull(BILL_TYPE#413), NOT
(QUANT_B#399 = 0.000), NOT (BILL_TYPE#413 = ZF8), NOT (BILL_TYPE#413 = ZS1), NOT
(BILL_TYPE#413 = F8), NOT (BILL_TYPE#413 = ZP02), dynamicpruning#308662 308661)
Actual: Set(isnotnull(BILL_DATE#412), isnotnull(QUANT_B#399), (BILL_DATE#412 =
2022-11-01), isnotnull(BILL_TYPE#413), isnotnull(MATERIAL#370), NOT (QUANT_B#399 =
0.000), NOT (BILL_TYPE#413 = ZF8), NOT (BILL_TYPE#413 = ZS1), NOT (BILL_TYPE#413 =
F8), NOT (BILL_TYPE#413 = ZP02), DISTR_CHAN#364 IN (20,25), COMP_CODE#362 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#370 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#93592), DISTR_CHAN#93586 IN (20,25),
COMP_CODE#93584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#93634),
isnotnull(QUANT_B#93621), (BILL_DATE#93634 = 2022-11-19),
isnotnull(BILL_TYPE#93635), NOT (QUANT_B#93621 = 0.000), NOT (BILL_TYPE#93635 =
ZF8), NOT (BILL_TYPE#93635 = ZS1), NOT (BILL_TYPE#93635 = F8), NOT (BILL_TYPE#93635
= ZP02), dynamicpruning#308698 308697)
Actual: Set(isnotnull(BILL_DATE#93634), isnotnull(QUANT_B#93621), (BILL_DATE#93634
= 2022-11-19), isnotnull(BILL_TYPE#93635), isnotnull(MATERIAL#93592), NOT
(QUANT_B#93621 = 0.000), NOT (BILL_TYPE#93635 = ZF8), NOT (BILL_TYPE#93635 = ZS1),
NOT (BILL_TYPE#93635 = F8), NOT (BILL_TYPE#93635 = ZP02), DISTR_CHAN#93586 IN
(20,25), COMP_CODE#93584 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#93592 IN
dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#133081), DISTR_CHAN#133075 IN (20,25),
COMP_CODE#133073 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#133123),
isnotnull(QUANT_B#133110), (BILL_DATE#133123 = 2022-11-23),
isnotnull(BILL_TYPE#133124), NOT (QUANT_B#133110 = 0.000), NOT (BILL_TYPE#133124 =
ZF8), NOT (BILL_TYPE#133124 = ZS1), NOT (BILL_TYPE#133124 = F8), NOT
(BILL_TYPE#133124 = ZP02), dynamicpruning#308706 308705)
Actual: Set(isnotnull(BILL_DATE#133123), isnotnull(QUANT_B#133110),
(BILL_DATE#133123 = 2022-11-23), isnotnull(BILL_TYPE#133124),
isnotnull(MATERIAL#133081), NOT (QUANT_B#133110 = 0.000), NOT (BILL_TYPE#133124 =
ZF8), NOT (BILL_TYPE#133124 = ZS1), NOT (BILL_TYPE#133124 = F8), NOT
(BILL_TYPE#133124 = ZP02), DISTR_CHAN#133075 IN (20,25), COMP_CODE#133073 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#133081 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#155325), DISTR_CHAN#155319 IN (20,25),
COMP_CODE#155317 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#155367),
isnotnull(QUANT_B#155354), (BILL_DATE#155367 = 2022-11-25),
isnotnull(BILL_TYPE#155368), NOT (QUANT_B#155354 = 0.000), NOT (BILL_TYPE#155368 =
ZF8), NOT (BILL_TYPE#155368 = ZS1), NOT (BILL_TYPE#155368 = F8), NOT
(BILL_TYPE#155368 = ZP02), dynamicpruning#308710 308709)
Actual: Set(isnotnull(BILL_DATE#155367), isnotnull(QUANT_B#155354),
(BILL_DATE#155367 = 2022-11-25), isnotnull(BILL_TYPE#155368),
isnotnull(MATERIAL#155325), NOT (QUANT_B#155354 = 0.000), NOT (BILL_TYPE#155368 =
ZF8), NOT (BILL_TYPE#155368 = ZS1), NOT (BILL_TYPE#155368 = F8), NOT
(BILL_TYPE#155368 = ZP02), DISTR_CHAN#155319 IN (20,25), COMP_CODE#155317 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#155325 IN dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#3481), DISTR_CHAN#3475 IN (20,25), COMP_CODE#3473
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#3523),
isnotnull(QUANT_B#3510), (BILL_DATE#3523 = 2022-11-03), isnotnull(BILL_TYPE#3524),
NOT (QUANT_B#3510 = 0.000), NOT (BILL_TYPE#3524 = ZF8), NOT (BILL_TYPE#3524 = ZS1),
NOT (BILL_TYPE#3524 = F8), NOT (BILL_TYPE#3524 = ZP02), dynamicpruning#308666
308665)
Actual: Set(isnotnull(BILL_DATE#3523), isnotnull(QUANT_B#3510), (BILL_DATE#3523 =
2022-11-03), isnotnull(BILL_TYPE#3524), isnotnull(MATERIAL#3481), NOT (QUANT_B#3510
= 0.000), NOT (BILL_TYPE#3524 = ZF8), NOT (BILL_TYPE#3524 = ZS1), NOT
(BILL_TYPE#3524 = F8), NOT (BILL_TYPE#3524 = ZP02), DISTR_CHAN#3475 IN (20,25),
COMP_CODE#3473 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#3481 IN
dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#76477), DISTR_CHAN#76471 IN (20,25),
COMP_CODE#76469 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#76519),
isnotnull(QUANT_B#76506), (BILL_DATE#76519 = 2022-11-17),
isnotnull(BILL_TYPE#76520), NOT (QUANT_B#76506 = 0.000), NOT (BILL_TYPE#76520 =
ZF8), NOT (BILL_TYPE#76520 = ZS1), NOT (BILL_TYPE#76520 = F8), NOT (BILL_TYPE#76520
= ZP02), dynamicpruning#308694 308693)
Actual: Set(isnotnull(BILL_DATE#76519), isnotnull(QUANT_B#76506), (BILL_DATE#76519
= 2022-11-17), isnotnull(BILL_TYPE#76520), isnotnull(MATERIAL#76477), NOT
(QUANT_B#76506 = 0.000), NOT (BILL_TYPE#76520 = ZF8), NOT (BILL_TYPE#76520 = ZS1),
NOT (BILL_TYPE#76520 = F8), NOT (BILL_TYPE#76520 = ZP02), DISTR_CHAN#76471 IN
(20,25), COMP_CODE#76469 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#76477 IN
dynamicpruning#308662)
25/02/06 06:12:28 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#47181), DISTR_CHAN#47175 IN (20,25),
COMP_CODE#47173 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#47223),
isnotnull(QUANT_B#47210), (BILL_DATE#47223 = 2022-11-13),
isnotnull(BILL_TYPE#47224), NOT (QUANT_B#47210 = 0.000), NOT (BILL_TYPE#47224 =
ZF8), NOT (BILL_TYPE#47224 = ZS1), NOT (BILL_TYPE#47224 = F8), NOT (BILL_TYPE#47224
= ZP02), dynamicpruning#308686 308685)
Actual: Set(isnotnull(BILL_DATE#47223), isnotnull(QUANT_B#47210), (BILL_DATE#47223
= 2022-11-13), isnotnull(BILL_TYPE#47224), isnotnull(MATERIAL#47181), NOT
(QUANT_B#47210 = 0.000), NOT (BILL_TYPE#47224 = ZF8), NOT (BILL_TYPE#47224 = ZS1),
NOT (BILL_TYPE#47224 = F8), NOT (BILL_TYPE#47224 = ZP02), DISTR_CHAN#47175 IN
(20,25), COMP_CODE#47173 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#47181 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11274.278612 ms
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11283.436369 ms
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11375.994416 ms
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11400.618372 ms
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11040.689625 ms
25/02/06 06:12:29 INFO CodeGenerator: Code generated in 11299.098206 ms
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#191856), DISTR_CHAN#191850 IN (20,25),
COMP_CODE#191848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#191898),
isnotnull(QUANT_B#191885), (BILL_DATE#191898 = 2022-11-28),
isnotnull(BILL_TYPE#191899), NOT (QUANT_B#191885 = 0.000), NOT (BILL_TYPE#191899 =
ZF8), NOT (BILL_TYPE#191899 = ZS1), NOT (BILL_TYPE#191899 = F8), NOT
(BILL_TYPE#191899 = ZP02), dynamicpruning#308716 308715)
Actual: Set(isnotnull(BILL_DATE#191898), isnotnull(QUANT_B#191885),
(BILL_DATE#191898 = 2022-11-28), isnotnull(BILL_TYPE#191899),
isnotnull(MATERIAL#191856), NOT (QUANT_B#191885 = 0.000), NOT (BILL_TYPE#191899 =
ZF8), NOT (BILL_TYPE#191899 = ZS1), NOT (BILL_TYPE#191899 = F8), NOT
(BILL_TYPE#191899 = ZP02), DISTR_CHAN#191850 IN (20,25), COMP_CODE#191848 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#191856 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#143992), DISTR_CHAN#143986 IN (20,25),
COMP_CODE#143984 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#144034),
isnotnull(QUANT_B#144021), (BILL_DATE#144034 = 2022-11-24),
isnotnull(BILL_TYPE#144035), NOT (QUANT_B#144021 = 0.000), NOT (BILL_TYPE#144035 =
ZF8), NOT (BILL_TYPE#144035 = ZS1), NOT (BILL_TYPE#144035 = F8), NOT
(BILL_TYPE#144035 = ZP02), dynamicpruning#308708 308707)
Actual: Set(isnotnull(BILL_DATE#144034), isnotnull(QUANT_B#144021),
(BILL_DATE#144034 = 2022-11-24), isnotnull(BILL_TYPE#144035),
isnotnull(MATERIAL#143992), NOT (QUANT_B#144021 = 0.000), NOT (BILL_TYPE#144035 =
ZF8), NOT (BILL_TYPE#144035 = ZS1), NOT (BILL_TYPE#144035 = F8), NOT
(BILL_TYPE#144035 = ZP02), DISTR_CHAN#143986 IN (20,25), COMP_CODE#143984 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#143992 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#60985), DISTR_CHAN#60979 IN (20,25),
COMP_CODE#60977 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#61027),
isnotnull(QUANT_B#61014), (BILL_DATE#61027 = 2022-11-15),
isnotnull(BILL_TYPE#61028), NOT (QUANT_B#61014 = 0.000), NOT (BILL_TYPE#61028 =
ZF8), NOT (BILL_TYPE#61028 = ZS1), NOT (BILL_TYPE#61028 = F8), NOT (BILL_TYPE#61028
= ZP02), dynamicpruning#308690 308689)
Actual: Set(isnotnull(BILL_DATE#61027), isnotnull(QUANT_B#61014), (BILL_DATE#61027
= 2022-11-15), isnotnull(BILL_TYPE#61028), isnotnull(MATERIAL#60985), NOT
(QUANT_B#61014 = 0.000), NOT (BILL_TYPE#61028 = ZF8), NOT (BILL_TYPE#61028 = ZS1),
NOT (BILL_TYPE#61028 = F8), NOT (BILL_TYPE#61028 = ZP02), DISTR_CHAN#60979 IN
(20,25), COMP_CODE#60977 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#60985 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#20056), DISTR_CHAN#20050 IN (20,25),
COMP_CODE#20048 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#20098),
isnotnull(QUANT_B#20085), (BILL_DATE#20098 = 2022-11-08),
isnotnull(BILL_TYPE#20099), NOT (QUANT_B#20085 = 0.000), NOT (BILL_TYPE#20099 =
ZF8), NOT (BILL_TYPE#20099 = ZS1), NOT (BILL_TYPE#20099 = F8), NOT (BILL_TYPE#20099
= ZP02), dynamicpruning#308676 308675)
Actual: Set(isnotnull(BILL_DATE#20098), isnotnull(QUANT_B#20085), (BILL_DATE#20098
= 2022-11-08), isnotnull(BILL_TYPE#20099), isnotnull(MATERIAL#20056), NOT
(QUANT_B#20085 = 0.000), NOT (BILL_TYPE#20099 = ZF8), NOT (BILL_TYPE#20099 = ZS1),
NOT (BILL_TYPE#20099 = F8), NOT (BILL_TYPE#20099 = ZP02), DISTR_CHAN#20050 IN
(20,25), COMP_CODE#20048 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#20056 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#40912), DISTR_CHAN#40906 IN (20,25),
COMP_CODE#40904 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#40954),
isnotnull(QUANT_B#40941), (BILL_DATE#40954 = 2022-11-12),
isnotnull(BILL_TYPE#40955), NOT (QUANT_B#40941 = 0.000), NOT (BILL_TYPE#40955 =
ZF8), NOT (BILL_TYPE#40955 = ZS1), NOT (BILL_TYPE#40955 = F8), NOT (BILL_TYPE#40955
= ZP02), dynamicpruning#308684 308683)
Actual: Set(isnotnull(BILL_DATE#40954), isnotnull(QUANT_B#40941), (BILL_DATE#40954
= 2022-11-12), isnotnull(BILL_TYPE#40955), isnotnull(MATERIAL#40912), NOT
(QUANT_B#40941 = 0.000), NOT (BILL_TYPE#40955 = ZF8), NOT (BILL_TYPE#40955 = ZS1),
NOT (BILL_TYPE#40955 = F8), NOT (BILL_TYPE#40955 = ZP02), DISTR_CHAN#40906 IN
(20,25), COMP_CODE#40904 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#40912 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#84856), DISTR_CHAN#84850 IN (20,25),
COMP_CODE#84848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#84898),
isnotnull(QUANT_B#84885), (BILL_DATE#84898 = 2022-11-18),
isnotnull(BILL_TYPE#84899), NOT (QUANT_B#84885 = 0.000), NOT (BILL_TYPE#84899 =
ZF8), NOT (BILL_TYPE#84899 = ZS1), NOT (BILL_TYPE#84899 = F8), NOT (BILL_TYPE#84899
= ZP02), dynamicpruning#308696 308695)
Actual: Set(isnotnull(BILL_DATE#84898), isnotnull(QUANT_B#84885), (BILL_DATE#84898
= 2022-11-18), isnotnull(BILL_TYPE#84899), isnotnull(MATERIAL#84856), NOT
(QUANT_B#84885 = 0.000), NOT (BILL_TYPE#84899 = ZF8), NOT (BILL_TYPE#84899 = ZS1),
NOT (BILL_TYPE#84899 = F8), NOT (BILL_TYPE#84899 = ZP02), DISTR_CHAN#84850 IN
(20,25), COMP_CODE#84848 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#84856 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#204877), DISTR_CHAN#204871 IN (20,25),
COMP_CODE#204869 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#204919),
isnotnull(QUANT_B#204906), (BILL_DATE#204919 = 2022-11-29),
isnotnull(BILL_TYPE#204920), NOT (QUANT_B#204906 = 0.000), NOT (BILL_TYPE#204920 =
ZF8), NOT (BILL_TYPE#204920 = ZS1), NOT (BILL_TYPE#204920 = F8), NOT
(BILL_TYPE#204920 = ZP02), dynamicpruning#308718 308717)
Actual: Set(isnotnull(BILL_DATE#204919), isnotnull(QUANT_B#204906),
(BILL_DATE#204919 = 2022-11-29), isnotnull(BILL_TYPE#204920),
isnotnull(MATERIAL#204877), NOT (QUANT_B#204906 = 0.000), NOT (BILL_TYPE#204920 =
ZF8), NOT (BILL_TYPE#204920 = ZS1), NOT (BILL_TYPE#204920 = F8), NOT
(BILL_TYPE#204920 = ZP02), DISTR_CHAN#204871 IN (20,25), COMP_CODE#204869 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#204877 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#24637), DISTR_CHAN#24631 IN (20,25),
COMP_CODE#24629 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#24679),
isnotnull(QUANT_B#24666), (BILL_DATE#24679 = 2022-11-09),
isnotnull(BILL_TYPE#24680), NOT (QUANT_B#24666 = 0.000), NOT (BILL_TYPE#24680 =
ZF8), NOT (BILL_TYPE#24680 = ZS1), NOT (BILL_TYPE#24680 = F8), NOT (BILL_TYPE#24680
= ZP02), dynamicpruning#308678 308677)
Actual: Set(isnotnull(BILL_DATE#24679), isnotnull(QUANT_B#24666), (BILL_DATE#24679
= 2022-11-09), isnotnull(BILL_TYPE#24680), isnotnull(MATERIAL#24637), NOT
(QUANT_B#24666 = 0.000), NOT (BILL_TYPE#24680 = ZF8), NOT (BILL_TYPE#24680 = ZS1),
NOT (BILL_TYPE#24680 = F8), NOT (BILL_TYPE#24680 = ZP02), DISTR_CHAN#24631 IN
(20,25), COMP_CODE#24629 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#24637 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#179257), DISTR_CHAN#179251 IN (20,25),
COMP_CODE#179249 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#179299),
isnotnull(QUANT_B#179286), (BILL_DATE#179299 = 2022-11-27),
isnotnull(BILL_TYPE#179300), NOT (QUANT_B#179286 = 0.000), NOT (BILL_TYPE#179300 =
ZF8), NOT (BILL_TYPE#179300 = ZS1), NOT (BILL_TYPE#179300 = F8), NOT
(BILL_TYPE#179300 = ZP02), dynamicpruning#308714 308713)
Actual: Set(isnotnull(BILL_DATE#179299), isnotnull(QUANT_B#179286),
(BILL_DATE#179299 = 2022-11-27), isnotnull(BILL_TYPE#179300),
isnotnull(MATERIAL#179257), NOT (QUANT_B#179286 = 0.000), NOT (BILL_TYPE#179300 =
ZF8), NOT (BILL_TYPE#179300 = ZS1), NOT (BILL_TYPE#179300 = F8), NOT
(BILL_TYPE#179300 = ZP02), DISTR_CHAN#179251 IN (20,25), COMP_CODE#179249 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#179257 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#35065), DISTR_CHAN#35059 IN (20,25),
COMP_CODE#35057 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#35107),
isnotnull(QUANT_B#35094), (BILL_DATE#35107 = 2022-11-11),
isnotnull(BILL_TYPE#35108), NOT (QUANT_B#35094 = 0.000), NOT (BILL_TYPE#35108 =
ZF8), NOT (BILL_TYPE#35108 = ZS1), NOT (BILL_TYPE#35108 = F8), NOT (BILL_TYPE#35108
= ZP02), dynamicpruning#308682 308681)
Actual: Set(isnotnull(BILL_DATE#35107), isnotnull(QUANT_B#35094), (BILL_DATE#35107
= 2022-11-11), isnotnull(BILL_TYPE#35108), isnotnull(MATERIAL#35065), NOT
(QUANT_B#35094 = 0.000), NOT (BILL_TYPE#35108 = ZF8), NOT (BILL_TYPE#35108 = ZS1),
NOT (BILL_TYPE#35108 = F8), NOT (BILL_TYPE#35108 = ZP02), DISTR_CHAN#35059 IN
(20,25), COMP_CODE#35057 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#35065 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#218320), DISTR_CHAN#218314 IN (20,25),
COMP_CODE#218312 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#218362),
isnotnull(QUANT_B#218349), (BILL_DATE#218362 = 2022-11-30),
isnotnull(BILL_TYPE#218363), NOT (QUANT_B#218349 = 0.000), NOT (BILL_TYPE#218363 =
ZF8), NOT (BILL_TYPE#218363 = ZS1), NOT (BILL_TYPE#218363 = F8), NOT
(BILL_TYPE#218363 = ZP02), dynamicpruning#308720 308719)
Actual: Set(isnotnull(BILL_DATE#218362), isnotnull(QUANT_B#218349),
(BILL_DATE#218362 = 2022-11-30), isnotnull(BILL_TYPE#218363),
isnotnull(MATERIAL#218320), NOT (QUANT_B#218349 = 0.000), NOT (BILL_TYPE#218363 =
ZF8), NOT (BILL_TYPE#218363 = ZS1), NOT (BILL_TYPE#218363 = F8), NOT
(BILL_TYPE#218363 = ZP02), DISTR_CHAN#218314 IN (20,25), COMP_CODE#218312 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#218320 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#15897), DISTR_CHAN#15891 IN (20,25),
COMP_CODE#15889 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#15939),
isnotnull(QUANT_B#15926), (BILL_DATE#15939 = 2022-11-07),
isnotnull(BILL_TYPE#15940), NOT (QUANT_B#15926 = 0.000), NOT (BILL_TYPE#15940 =
ZF8), NOT (BILL_TYPE#15940 = ZS1), NOT (BILL_TYPE#15940 = F8), NOT (BILL_TYPE#15940
= ZP02), dynamicpruning#308674 308673)
Actual: Set(isnotnull(BILL_DATE#15939), isnotnull(QUANT_B#15926), (BILL_DATE#15939
= 2022-11-07), isnotnull(BILL_TYPE#15940), isnotnull(MATERIAL#15897), NOT
(QUANT_B#15926 = 0.000), NOT (BILL_TYPE#15940 = ZF8), NOT (BILL_TYPE#15940 = ZS1),
NOT (BILL_TYPE#15940 = F8), NOT (BILL_TYPE#15940 = ZP02), DISTR_CHAN#15891 IN
(20,25), COMP_CODE#15889 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#15897 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#112525), DISTR_CHAN#112519 IN (20,25),
COMP_CODE#112517 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#112567),
isnotnull(QUANT_B#112554), (BILL_DATE#112567 = 2022-11-21),
isnotnull(BILL_TYPE#112568), NOT (QUANT_B#112554 = 0.000), NOT (BILL_TYPE#112568 =
ZF8), NOT (BILL_TYPE#112568 = ZS1), NOT (BILL_TYPE#112568 = F8), NOT
(BILL_TYPE#112568 = ZP02), dynamicpruning#308702 308701)
Actual: Set(isnotnull(BILL_DATE#112567), isnotnull(QUANT_B#112554),
(BILL_DATE#112567 = 2022-11-21), isnotnull(BILL_TYPE#112568),
isnotnull(MATERIAL#112525), NOT (QUANT_B#112554 = 0.000), NOT (BILL_TYPE#112568 =
ZF8), NOT (BILL_TYPE#112568 = ZS1), NOT (BILL_TYPE#112568 = F8), NOT
(BILL_TYPE#112568 = ZP02), DISTR_CHAN#112519 IN (20,25), COMP_CODE#112517 INSET
1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600, 7700,
7800, 7900, 8000, 8100, 8200, MATERIAL#112525 IN dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#8845), DISTR_CHAN#8839 IN (20,25), COMP_CODE#8837
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#8887),
isnotnull(QUANT_B#8874), (BILL_DATE#8887 = 2022-11-05), isnotnull(BILL_TYPE#8888),
NOT (QUANT_B#8874 = 0.000), NOT (BILL_TYPE#8888 = ZF8), NOT (BILL_TYPE#8888 = ZS1),
NOT (BILL_TYPE#8888 = F8), NOT (BILL_TYPE#8888 = ZP02), dynamicpruning#308670
308669)
Actual: Set(isnotnull(BILL_DATE#8887), isnotnull(QUANT_B#8874), (BILL_DATE#8887 =
2022-11-05), isnotnull(BILL_TYPE#8888), isnotnull(MATERIAL#8845), NOT (QUANT_B#8874
= 0.000), NOT (BILL_TYPE#8888 = ZF8), NOT (BILL_TYPE#8888 = ZS1), NOT
(BILL_TYPE#8888 = F8), NOT (BILL_TYPE#8888 = ZP02), DISTR_CHAN#8839 IN (20,25),
COMP_CODE#8837 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#8845 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#68520), DISTR_CHAN#68514 IN (20,25),
COMP_CODE#68512 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#68562),
isnotnull(QUANT_B#68549), (BILL_DATE#68562 = 2022-11-16),
isnotnull(BILL_TYPE#68563), NOT (QUANT_B#68549 = 0.000), NOT (BILL_TYPE#68563 =
ZF8), NOT (BILL_TYPE#68563 = ZS1), NOT (BILL_TYPE#68563 = F8), NOT (BILL_TYPE#68563
= ZP02), dynamicpruning#308692 308691)
Actual: Set(isnotnull(BILL_DATE#68562), isnotnull(QUANT_B#68549), (BILL_DATE#68562
= 2022-11-16), isnotnull(BILL_TYPE#68563), isnotnull(MATERIAL#68520), NOT
(QUANT_B#68549 = 0.000), NOT (BILL_TYPE#68563 = ZF8), NOT (BILL_TYPE#68563 = ZS1),
NOT (BILL_TYPE#68563 = F8), NOT (BILL_TYPE#68563 = ZP02), DISTR_CHAN#68514 IN
(20,25), COMP_CODE#68512 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#68520 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#12160), DISTR_CHAN#12154 IN (20,25),
COMP_CODE#12152 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#12202),
isnotnull(QUANT_B#12189), (BILL_DATE#12202 = 2022-11-06),
isnotnull(BILL_TYPE#12203), NOT (QUANT_B#12189 = 0.000), NOT (BILL_TYPE#12203 =
ZF8), NOT (BILL_TYPE#12203 = ZS1), NOT (BILL_TYPE#12203 = F8), NOT (BILL_TYPE#12203
= ZP02), dynamicpruning#308672 308671)
Actual: Set(isnotnull(BILL_DATE#12202), isnotnull(QUANT_B#12189), (BILL_DATE#12202
= 2022-11-06), isnotnull(BILL_TYPE#12203), isnotnull(MATERIAL#12160), NOT
(QUANT_B#12189 = 0.000), NOT (BILL_TYPE#12203 = ZF8), NOT (BILL_TYPE#12203 = ZS1),
NOT (BILL_TYPE#12203 = F8), NOT (BILL_TYPE#12203 = ZP02), DISTR_CHAN#12154 IN
(20,25), COMP_CODE#12152 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#12160 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO TaskSetManager: Starting task 0.0 in stage 244.0 (TID 1113)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#5952), DISTR_CHAN#5946 IN (20,25), COMP_CODE#5944
INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450, 7500, 7550, 7600,
7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#5994),
isnotnull(QUANT_B#5981), (BILL_DATE#5994 = 2022-11-04), isnotnull(BILL_TYPE#5995),
NOT (QUANT_B#5981 = 0.000), NOT (BILL_TYPE#5995 = ZF8), NOT (BILL_TYPE#5995 = ZS1),
NOT (BILL_TYPE#5995 = F8), NOT (BILL_TYPE#5995 = ZP02), dynamicpruning#308668
308667)
Actual: Set(isnotnull(BILL_DATE#5994), isnotnull(QUANT_B#5981), (BILL_DATE#5994 =
2022-11-04), isnotnull(BILL_TYPE#5995), isnotnull(MATERIAL#5952), NOT (QUANT_B#5981
= 0.000), NOT (BILL_TYPE#5995 = ZF8), NOT (BILL_TYPE#5995 = ZS1), NOT
(BILL_TYPE#5995 = F8), NOT (BILL_TYPE#5995 = ZP02), DISTR_CHAN#5946 IN (20,25),
COMP_CODE#5944 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#5952 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO TaskSetManager: Finished task 4.0 in stage 238.0 (TID 1090)
in 12094 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:12:29 INFO MemoryStore: Block broadcast_74 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#29640), DISTR_CHAN#29634 IN (20,25),
COMP_CODE#29632 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#29682),
isnotnull(QUANT_B#29669), (BILL_DATE#29682 = 2022-11-10),
isnotnull(BILL_TYPE#29683), NOT (QUANT_B#29669 = 0.000), NOT (BILL_TYPE#29683 =
ZF8), NOT (BILL_TYPE#29683 = ZS1), NOT (BILL_TYPE#29683 = F8), NOT (BILL_TYPE#29683
= ZP02), dynamicpruning#308680 308679)
Actual: Set(isnotnull(BILL_DATE#29682), isnotnull(QUANT_B#29669), (BILL_DATE#29682
= 2022-11-10), isnotnull(BILL_TYPE#29683), isnotnull(MATERIAL#29640), NOT
(QUANT_B#29669 = 0.000), NOT (BILL_TYPE#29683 = ZF8), NOT (BILL_TYPE#29683 = ZS1),
NOT (BILL_TYPE#29683 = F8), NOT (BILL_TYPE#29683 = ZP02), DISTR_CHAN#29634 IN
(20,25), COMP_CODE#29632 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#29640 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:29 INFO PreparedDeltaFileIndex:
Prepared scan does not match actual filters. Reselecting files to query.
Prepared: Set(isnotnull(MATERIAL#53872), DISTR_CHAN#53866 IN (20,25),
COMP_CODE#53864 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200, 7400, 7450,
7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, isnotnull(BILL_DATE#53914),
isnotnull(QUANT_B#53901), (BILL_DATE#53914 = 2022-11-14),
isnotnull(BILL_TYPE#53915), NOT (QUANT_B#53901 = 0.000), NOT (BILL_TYPE#53915 =
ZF8), NOT (BILL_TYPE#53915 = ZS1), NOT (BILL_TYPE#53915 = F8), NOT (BILL_TYPE#53915
= ZP02), dynamicpruning#308688 308687)
Actual: Set(isnotnull(BILL_DATE#53914), isnotnull(QUANT_B#53901), (BILL_DATE#53914
= 2022-11-14), isnotnull(BILL_TYPE#53915), isnotnull(MATERIAL#53872), NOT
(QUANT_B#53901 = 0.000), NOT (BILL_TYPE#53915 = ZF8), NOT (BILL_TYPE#53915 = ZS1),
NOT (BILL_TYPE#53915 = F8), NOT (BILL_TYPE#53915 = ZP02), DISTR_CHAN#53866 IN
(20,25), COMP_CODE#53864 INSET 1000, 2000, 3000, 3200, 3400, 3600, 7000, 7200,
7400, 7450, 7500, 7550, 7600, 7700, 7800, 7900, 8000, 8100, 8200, MATERIAL#53872 IN
dynamicpruning#308662)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_25_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO TaskSetManager: Finished task 3.0 in stage 238.0 (TID 1089)
in 39993 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:12:29 INFO TaskSchedulerImpl: Removed TaskSet 238.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:29 INFO TaskSetManager: Starting task 1.0 in stage 244.0 (TID 1114)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:12:29 INFO DAGScheduler: ShuffleMapStage 238 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 110.002 s
25/02/06 06:12:29 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:29 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 240, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:12:29 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:29 INFO DAGScheduler: failed: Set()
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_7_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_2_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_11_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.232.44:37705 in memory (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:12:29 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_8_piece0 on
10.183.232.44:37705 in memory (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:40 INFO ProgressReporter$: Reporting partial results for running
commands: 5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9
[495 occurrences]
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_38_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_23_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_53_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_61_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_44_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.229.123:38441 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_56_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_33_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_22_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_59_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_11_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_2_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_21_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_3_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_13_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_7_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_53_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_23_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_39_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_34_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_61_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_59_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_53_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_52_piece0 on
10.183.233.191:42923 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_25_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_21_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_61_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_56_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_8_piece0 on
10.183.233.191:42923 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_59_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_11_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_23_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_61_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_38_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_28_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_39_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_34_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_8_piece0 on
10.183.232.45:45579 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_11_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_13_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_39_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_34_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_28_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_45_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_38_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_23_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_56_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_53_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_7_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_25_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_2_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_67_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_21_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_3_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_28_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_34_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_7_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_23_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_45_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_24_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_12_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_58_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_25_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_21_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_53_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_13_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_48_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_54_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_38_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_30_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_38_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_39_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_36_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_45_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_28_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO SparkContext: Created broadcast 74 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_69_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 77 to 10.183.229.123:55848
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 77 to 10.183.232.45:60256
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 77 to 10.183.249.197:36834
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 78 to 10.183.233.191:55538
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 77 to 10.183.233.191:55538
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 77 to 10.183.246.72:39896
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 78 to 10.183.246.72:39896
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_69_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_70_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_68_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_70_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_69_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_70_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_69_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 78 to 10.183.229.123:55848
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_70_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_68_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 78 to 10.183.249.197:36834
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_68_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 79 to 10.183.246.72:39896
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_64_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_64_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_64_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 79 to 10.183.233.191:55538
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 78 to 10.183.232.45:60256
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_9_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_9_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_9_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_14_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_14_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_14_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_14_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_62_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_62_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_62_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_62_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_10_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_10_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_10_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_10_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_31_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_31_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_31_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_69_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO TaskSetManager: Finished task 8.0 in stage 240.0 (TID 1103)
in 23740 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:12:40 INFO TaskSetManager: Starting task 2.0 in stage 244.0 (TID 1115)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_31_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_49_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_49_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_49_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_49_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_70_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_71_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_64_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_43_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_47_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_9_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_17_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_41_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_14_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_15_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_15_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_65_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_15_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_15_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_65_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_55_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_55_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_5_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_65_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_55_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_5_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 79 to 10.183.249.197:36834
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO TaskSetManager: Finished task 8.0 in stage 242.0 (TID 1112)
in 12055 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:12:40 INFO TaskSetManager: Starting task 3.0 in stage 244.0 (TID 1116)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_55_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_65_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_42_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_37_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_15_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_46_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_5_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_16_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_68_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_40_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_31_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_49_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_50_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_62_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_10_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_32_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_66_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_29_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:40 INFO BlockManagerInfo: Removed broadcast_26_piece0 on
10.183.232.44:37705 in memory (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_26_piece0 on
10.183.229.123:38441 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_51_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_57_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_51_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_26_piece0 on
10.183.249.197:40023 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_4_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_51_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_26_piece0 on
10.183.232.45:45579 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_60_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_4_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_60_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_51_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_26_piece0 on
10.183.246.72:43231 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_60_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_51_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_60_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_4_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_35_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_35_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_35_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_27_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_35_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO TaskSetManager: Finished task 4.0 in stage 240.0 (TID 1099)
in 24076 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 4.0 in stage 244.0 (TID 1117)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_20_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_20_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_20_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_20_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_35_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_20_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_63_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_63_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_63_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_63_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Removed broadcast_63_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 5.0 in stage 244.0 (TID 1118)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 2.0 in stage 242.0 (TID 1106)
in 24135 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:12:41 INFO TaskSetManager: Finished task 5.0 in stage 240.0 (TID 1100)
in 24170 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 6.0 in stage 244.0 (TID 1119)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 2.0 in stage 240.0 (TID 1097)
in 24174 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 7.0 in stage 244.0 (TID 1120)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 79 to 10.183.232.45:60256
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:41 INFO TaskSetManager: Starting task 8.0 in stage 244.0 (TID 1121)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 1.0 in stage 240.0 (TID 1096)
in 24195 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 0.0 in stage 246.0 (TID 1122)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 0.0 in stage 240.0 (TID 1095)
in 24212 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:12:41 INFO MemoryStore: Block broadcast_75 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 79 to 10.183.229.123:55848
25/02/06 06:12:41 INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in
memory (estimated size 38.6 KiB, free 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on
10.183.232.44:37705 (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO SparkContext: Created broadcast 75 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on
10.183.233.191:42923 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 48 to 10.183.233.191:55538
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:41 INFO TaskSetManager: Starting task 1.0 in stage 246.0 (TID 1123)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 1.0 in stage 242.0 (TID 1105)
in 24239 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on
10.183.249.197:40023 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 48 to 10.183.249.197:36834
25/02/06 06:12:41 INFO TaskSetManager: Starting task 2.0 in stage 246.0 (TID 1124)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 0.0 in stage 242.0 (TID 1104)
in 24256 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:41 INFO TaskSetManager: Starting task 3.0 in stage 246.0 (TID 1125)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 3.0 in stage 240.0 (TID 1098)
in 24284 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 4.0 in stage 246.0 (TID 1126)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 6.0 in stage 240.0 (TID 1101)
in 24336 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:12:41 INFO TaskSetManager: Finished task 6.0 in stage 242.0 (TID 1110)
in 23329 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 5.0 in stage 246.0 (TID 1127)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on
10.183.229.123:38441 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 48 to 10.183.229.123:55848
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on
10.183.232.45:45579 (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 48 to 10.183.232.45:60256
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:41 INFO TaskSetManager: Starting task 6.0 in stage 246.0 (TID 1128)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 7.0 in stage 242.0 (TID 1111)
in 12813 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 7.0 in stage 246.0 (TID 1129)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 0.0 in stage 244.0 (TID 1113)
in 12334 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 8.0 in stage 246.0 (TID 1130)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Starting task 0.0 in stage 248.0 (TID 1131)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 5.0 in stage 242.0 (TID 1109)
in 23416 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:12:41 INFO TaskSetManager: Finished task 3.0 in stage 244.0 (TID 1116)
in 637 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:12:41 INFO MemoryStore: Block broadcast_76 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:41 INFO MemoryStore: Block broadcast_76_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:41 INFO SparkContext: Created broadcast 76 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 52 to 10.183.246.72:39896
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:41 INFO TaskSetManager: Finished task 3.0 in stage 242.0 (TID 1107)
in 23475 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:12:41 INFO TaskSetManager: Starting task 1.0 in stage 248.0 (TID 1132)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Starting task 2.0 in stage 248.0 (TID 1133)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 1.0 in stage 244.0 (TID 1114)
in 12237 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 52 to 10.183.229.123:55848
25/02/06 06:12:41 INFO TaskSetManager: Starting task 3.0 in stage 248.0 (TID 1134)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:12:41 INFO TaskSetManager: Finished task 7.0 in stage 240.0 (TID 1102)
in 24530 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:12:41 INFO TaskSchedulerImpl: Removed TaskSet 240.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:41 INFO DAGScheduler: ShuffleMapStage 240 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 120.259 s
25/02/06 06:12:41 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:41 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 242,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 244, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:41 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:41 INFO DAGScheduler: failed: Set()
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:41 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 52 to 10.183.232.45:60256
25/02/06 06:12:41 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:12:50 INFO TaskSetManager: Finished task 6.0 in stage 244.0 (TID 1119)
in 9323 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 0.0 in stage 246.0 (TID 1122)
in 9283 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 2.0 in stage 246.0 (TID 1124)
in 9239 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 7.0 in stage 246.0 (TID 1129)
in 9071 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 8.0 in stage 246.0 (TID 1130)
in 9067 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 7.0 in stage 244.0 (TID 1120)
in 9321 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 4.0 in stage 242.0 (TID 1108)
in 32490 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:12:50 INFO TaskSchedulerImpl: Removed TaskSet 242.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:50 INFO TaskSetManager: Finished task 5.0 in stage 246.0 (TID 1127)
in 9153 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:50 INFO TaskSetManager: Starting task 4.0 in stage 248.0 (TID 1135)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 5.0 in stage 248.0 (TID 1136)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 6.0 in stage 248.0 (TID 1137)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:12:50 INFO DAGScheduler: ShuffleMapStage 242 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 128.022 s
25/02/06 06:12:50 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:50 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 246, ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 311,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:50 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:50 INFO DAGScheduler: failed: Set()
25/02/06 06:12:50 INFO TaskSetManager: Starting task 7.0 in stage 248.0 (TID 1138)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 8.0 in stage 248.0 (TID 1139)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 0.0 in stage 250.0 (TID 1140)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 1.0 in stage 250.0 (TID 1141)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 2.0 in stage 250.0 (TID 1142)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 3.0 in stage 250.0 (TID 1143)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 4.0 in stage 250.0 (TID 1144)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 5.0 in stage 250.0 (TID 1145)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 6.0 in stage 250.0 (TID 1146)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Finished task 1.0 in stage 248.0 (TID 1132)
in 9023 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 4.0 in stage 246.0 (TID 1126)
in 9169 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:12:50 INFO MemoryStore: Block broadcast_77 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 3.0 in stage 246.0 (TID 1125)
in 9222 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 2.0 in stage 244.0 (TID 1115)
in 9761 ms on 10.183.249.197 (executor 7) (6/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 3.0 in stage 248.0 (TID 1134)
in 8976 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 1.0 in stage 246.0 (TID 1123)
in 9266 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 8.0 in stage 244.0 (TID 1121)
in 9311 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 6.0 in stage 246.0 (TID 1128)
in 9125 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:12:50 INFO TaskSchedulerImpl: Removed TaskSet 246.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:50 INFO TaskSetManager: Starting task 7.0 in stage 250.0 (TID 1147)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 8.0 in stage 250.0 (TID 1148)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 0.0 in stage 252.0 (TID 1149)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 1.0 in stage 252.0 (TID 1150)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:12:50 INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:50 INFO DAGScheduler: ShuffleMapStage 246 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 122.470 s
25/02/06 06:12:50 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:50 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 244, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:50 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:50 INFO DAGScheduler: failed: Set()
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO SparkContext: Created broadcast 77 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:50 INFO MemoryStore: Block broadcast_78 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:50 INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 5.0 in stage 244.0 (TID 1118)
in 9378 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 4.0 in stage 244.0 (TID 1117)
in 9440 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:12:50 INFO TaskSchedulerImpl: Removed TaskSet 244.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:50 INFO SparkContext: Created broadcast 78 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:50 INFO TaskSetManager: Finished task 0.0 in stage 248.0 (TID 1131)
in 9088 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO TaskSetManager: Finished task 2.0 in stage 248.0 (TID 1133)
in 9028 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO TaskSetManager: Starting task 2.0 in stage 252.0 (TID 1151)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 3.0 in stage 252.0 (TID 1152)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 4.0 in stage 252.0 (TID 1153)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Starting task 5.0 in stage 252.0 (TID 1154)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 52 to 10.183.233.191:55538
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 54 to 10.183.232.45:60256
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 54 to 10.183.249.197:36834
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 82 to 10.183.246.72:39896
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 82 to 10.183.249.197:36834
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:50 INFO DAGScheduler: ShuffleMapStage 244 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 125.122 s
25/02/06 06:12:50 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:50 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 248, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262,
ShuffleMapStage 254)
25/02/06 06:12:50 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:50 INFO DAGScheduler: failed: Set()
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 54 to 10.183.229.123:55848
25/02/06 06:12:50 INFO BlockManagerInfo: Removed broadcast_72_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:50 INFO TaskSetManager: Starting task 6.0 in stage 252.0 (TID 1155)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:12:50 INFO TaskSetManager: Finished task 8.0 in stage 248.0 (TID 1139)
in 376 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:12:50 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 82 to 10.183.232.45:60256
25/02/06 06:12:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:50 INFO TaskSetManager: Finished task 8.0 in stage 250.0 (TID 1148)
in 418 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:12:50 INFO TaskSetManager: Starting task 7.0 in stage 252.0 (TID 1156)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 6.0 in stage 248.0 (TID 1137)
in 782 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 8.0 in stage 252.0 (TID 1157)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 82 to 10.183.233.191:55538
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:51 INFO TaskSetManager: Finished task 3.0 in stage 252.0 (TID 1152)
in 795 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 0.0 in stage 254.0 (TID 1158)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_79 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_79_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:51 INFO SparkContext: Created broadcast 79 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 75 to 10.183.246.72:39896
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:51 INFO TaskSetManager: Starting task 1.0 in stage 254.0 (TID 1159)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 4.0 in stage 248.0 (TID 1135)
in 851 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 75 to 10.183.233.191:55538
25/02/06 06:12:51 INFO TaskSetManager: Finished task 7.0 in stage 248.0 (TID 1138)
in 863 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 2.0 in stage 254.0 (TID 1160)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:51 INFO TaskSetManager: Finished task 5.0 in stage 252.0 (TID 1154)
in 844 ms on 10.183.246.72 (executor 6) (2/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 3.0 in stage 254.0 (TID 1161)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Starting task 4.0 in stage 254.0 (TID 1162)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 1.0 in stage 252.0 (TID 1150)
in 914 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 75 to 10.183.249.197:36834
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:51 INFO TaskSetManager: Finished task 2.0 in stage 252.0 (TID 1151)
in 930 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 5.0 in stage 254.0 (TID 1163)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 1.0 in stage 250.0 (TID 1141)
in 956 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 6.0 in stage 254.0 (TID 1164)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Starting task 7.0 in stage 254.0 (TID 1165)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 2.0 in stage 250.0 (TID 1142)
in 957 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 75 to 10.183.232.45:60256
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:51 INFO TaskSetManager: Starting task 8.0 in stage 254.0 (TID 1166)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 5.0 in stage 248.0 (TID 1136)
in 983 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:12:51 INFO TaskSchedulerImpl: Removed TaskSet 248.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:51 INFO DAGScheduler: ShuffleMapStage 248 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 121.605 s
25/02/06 06:12:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:51 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 252, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:51 INFO DAGScheduler: failed: Set()
25/02/06 06:12:51 INFO TaskSetManager: Finished task 4.0 in stage 252.0 (TID 1153)
in 987 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 0.0 in stage 256.0 (TID 1167)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_80 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:51 INFO SparkContext: Created broadcast 80 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO TaskSetManager: Finished task 0.0 in stage 250.0 (TID 1140)
in 1024 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 1.0 in stage 256.0 (TID 1168)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 80 to 10.183.246.72:39896
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 80 to 10.183.232.45:60256
25/02/06 06:12:51 INFO TaskSetManager: Finished task 5.0 in stage 250.0 (TID 1145)
in 1041 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 2.0 in stage 256.0 (TID 1169)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO TaskSetManager: Finished task 3.0 in stage 250.0 (TID 1143)
in 1055 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 3.0 in stage 256.0 (TID 1170)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 80 to 10.183.229.123:55848
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:51 INFO TaskSetManager: Starting task 4.0 in stage 256.0 (TID 1171)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 6.0 in stage 250.0 (TID 1146)
in 1080 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 5.0 in stage 256.0 (TID 1172)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 7.0 in stage 250.0 (TID 1147)
in 1159 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:12:51 INFO TaskSetManager: Finished task 8.0 in stage 252.0 (TID 1157)
in 387 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 6.0 in stage 256.0 (TID 1173)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 7.0 in stage 256.0 (TID 1174)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 6.0 in stage 252.0 (TID 1155)
in 805 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 80 to 10.183.233.191:55538
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 80 to 10.183.249.197:36834
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:51 INFO TaskSetManager: Starting task 8.0 in stage 256.0 (TID 1175)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:51 INFO TaskSetManager: Finished task 0.0 in stage 252.0 (TID 1149)
in 1201 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:12:51 INFO TaskSetManager: Starting task 0.0 in stage 258.0 (TID 1176)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:12:51 INFO TaskSetManager: Finished task 8.0 in stage 254.0 (TID 1166)
in 343 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_81 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO MemoryStore: Block broadcast_81_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:12:51 INFO SparkContext: Created broadcast 81 from broadcast at
TaskSetManager.scala:711
25/02/06 06:12:51 INFO TaskSetManager: Starting task 1.0 in stage 258.0 (TID 1177)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO TaskSetManager: Finished task 7.0 in stage 252.0 (TID 1156)
in 920 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:12:51 INFO TaskSchedulerImpl: Removed TaskSet 252.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:51 INFO DAGScheduler: ShuffleMapStage 252 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 117.909 s
25/02/06 06:12:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:51 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 250,
ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 311,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305,
ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:51 INFO DAGScheduler: failed: Set()
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 81 to 10.183.233.191:55538
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 81 to 10.183.249.197:36834
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:12:51 INFO TaskSetManager: Finished task 4.0 in stage 250.0 (TID 1144)
in 1379 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:12:51 INFO TaskSchedulerImpl: Removed TaskSet 250.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:12:51 INFO TaskSetManager: Starting task 2.0 in stage 258.0 (TID 1178)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:12:51 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 81 to 10.183.229.123:55848
25/02/06 06:12:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:12:51 INFO DAGScheduler: ShuffleMapStage 250 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 119.939 s
25/02/06 06:12:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:12:51 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 302,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262, ShuffleMapStage 254)
25/02/06 06:12:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:12:51 INFO DAGScheduler: failed: Set()
25/02/06 06:13:03 WARN DBRDebuggerEventReporter: Driver/10.183.232.44 paused the
JVM process 66 seconds during the past 120 seconds (55.34%) because of GC.
25/02/06 06:13:03 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:03 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:03 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:03 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_75_piece0 on
10.183.229.123:38441 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_75_piece0 on
10.183.233.191:42923 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_77_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_76_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_76_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_78_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_75_piece0 on
10.183.232.44:37705 in memory (size: 38.6 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_77_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO TaskSetManager: Starting task 3.0 in stage 258.0 (TID 1179)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 4.0 in stage 258.0 (TID 1180)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 5.0 in stage 258.0 (TID 1181)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Finished task 4.0 in stage 256.0 (TID 1171)
in 11861 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 2.0 in stage 258.0 (TID 1178)
in 11561 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 2.0 in stage 256.0 (TID 1169)
in 11899 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 2.0 in stage 254.0 (TID 1160)
in 12084 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 3.0 in stage 256.0 (TID 1170)
in 11888 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 0.0 in stage 258.0 (TID 1176)
in 11625 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 1.0 in stage 254.0 (TID 1159)
in 12099 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:13:03 INFO TaskSetManager: Starting task 6.0 in stage 258.0 (TID 1182)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 7.0 in stage 258.0 (TID 1183)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 8.0 in stage 258.0 (TID 1184)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 0.0 in stage 260.0 (TID 1185)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 1.0 in stage 260.0 (TID 1186)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Finished task 6.0 in stage 256.0 (TID 1173)
in 11780 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 0.0 in stage 254.0 (TID 1158)
in 12134 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_76_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_78_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_75_piece0 on
10.183.249.197:40023 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_77_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_78_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 3.0 in stage 254.0 (TID 1161)
in 12140 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 5.0 in stage 254.0 (TID 1163)
in 12055 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 8.0 in stage 256.0 (TID 1175)
in 11803 ms on 10.183.249.197 (executor 7) (5/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 0.0 in stage 256.0 (TID 1167)
in 12000 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 4.0 in stage 254.0 (TID 1162)
in 12087 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 5.0 in stage 256.0 (TID 1172)
in 11847 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 1.0 in stage 258.0 (TID 1177)
in 11668 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 6.0 in stage 254.0 (TID 1164)
in 12053 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 7.0 in stage 254.0 (TID 1165)
in 12054 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:13:03 INFO TaskSchedulerImpl: Removed TaskSet 254.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:03 INFO TaskSetManager: Finished task 1.0 in stage 256.0 (TID 1168)
in 11987 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:13:03 INFO TaskSetManager: Finished task 7.0 in stage 256.0 (TID 1174)
in 11836 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:13:03 INFO TaskSchedulerImpl: Removed TaskSet 256.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_76_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_77_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_75_piece0 on
10.183.232.45:45579 in memory (size: 38.6 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_78_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO TaskSetManager: Starting task 2.0 in stage 260.0 (TID 1187)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 3.0 in stage 260.0 (TID 1188)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:13:03 INFO MemoryStore: Block broadcast_82 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:03 INFO TaskSetManager: Starting task 4.0 in stage 260.0 (TID 1189)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 5.0 in stage 260.0 (TID 1190)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_73_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_76_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_74_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Removed broadcast_78_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:13:03 INFO DAGScheduler: ShuffleMapStage 254 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 127.664 s
25/02/06 06:13:03 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO SparkContext: Created broadcast 82 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:03 INFO TaskSetManager: Starting task 6.0 in stage 260.0 (TID 1191)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 7.0 in stage 260.0 (TID 1192)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 8.0 in stage 260.0 (TID 1193)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 0.0 in stage 262.0 (TID 1194)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:13:03 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 256, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 302,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 264, ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341,
ShuffleMapStage 262)
25/02/06 06:13:03 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:03 INFO DAGScheduler: failed: Set()
25/02/06 06:13:03 INFO MemoryStore: Block broadcast_83 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:03 INFO TaskSetManager: Starting task 1.0 in stage 262.0 (TID 1195)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 2.0 in stage 262.0 (TID 1196)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 3.0 in stage 262.0 (TID 1197)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Starting task 4.0 in stage 262.0 (TID 1198)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:13:03 INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:03 INFO SparkContext: Created broadcast 83 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 83 to 10.183.246.72:39896
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 83 to 10.183.233.191:55538
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 83 to 10.183.249.197:36834
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 84 to 10.183.232.45:60256
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 84 to 10.183.249.197:36834
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 83 to 10.183.249.197:36834
25/02/06 06:13:03 INFO DAGScheduler: ShuffleMapStage 256 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 125.339 s
25/02/06 06:13:03 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:03 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 258, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ShuffleMapStage 292, ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 268, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264,
ShuffleMapStage 266, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 260, ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262)
25/02/06 06:13:03 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:03 INFO DAGScheduler: failed: Set()
25/02/06 06:13:03 INFO TaskSetManager: Starting task 5.0 in stage 262.0 (TID 1199)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:13:03 INFO TaskSetManager: Finished task 8.0 in stage 258.0 (TID 1184)
in 407 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:13:03 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 84 to 10.183.233.191:55538
25/02/06 06:13:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:04 INFO TaskSetManager: Starting task 6.0 in stage 262.0 (TID 1200)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:13:04 INFO TaskSetManager: Finished task 8.0 in stage 260.0 (TID 1193)
in 589 ms on 10.183.249.197 (executor 7) (1/9)
25/02/06 06:13:04 INFO TaskSetManager: Starting task 7.0 in stage 262.0 (TID 1201)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:13:04 INFO TaskSetManager: Finished task 4.0 in stage 258.0 (TID 1180)
in 803 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:13:04 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 84 to 10.183.229.123:55848
25/02/06 06:13:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:13:04 INFO TaskSetManager: Starting task 8.0 in stage 262.0 (TID 1202)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:13:04 INFO TaskSetManager: Finished task 7.0 in stage 258.0 (TID 1183)
in 846 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:13:04 INFO TaskSetManager: Finished task 0.0 in stage 260.0 (TID 1185)
in 911 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:13:04 INFO TaskSetManager: Starting task 0.0 in stage 264.0 (TID 1203)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:13:04 INFO MemoryStore: Block broadcast_84 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:04 INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:13:04 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:13:04 INFO SparkContext: Created broadcast 84 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:04 INFO TaskSetManager: Starting task 1.0 in stage 264.0 (TID 1204)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:13:04 INFO TaskSetManager: Finished task 6.0 in stage 260.0 (TID 1191)
in 804 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:13:04 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:04 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 86 to 10.183.233.191:55538
25/02/06 06:13:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 86 to 10.183.249.197:36834
25/02/06 06:13:12 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:12 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:12 INFO TaskSetManager: Finished task 3.0 in stage 258.0 (TID 1179)
in 9498 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:13:12 INFO TaskSetManager: Starting task 2.0 in stage 264.0 (TID 1205)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:13:12 INFO TaskSetManager: Starting task 3.0 in stage 264.0 (TID 1206)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:13:12 INFO TaskSetManager: Starting task 4.0 in stage 264.0 (TID 1207)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:13:12 INFO TaskSetManager: Starting task 5.0 in stage 264.0 (TID 1208)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:13:12 INFO TaskSetManager: Finished task 6.0 in stage 258.0 (TID 1182)
in 9501 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:13:12 INFO TaskSetManager: Finished task 3.0 in stage 260.0 (TID 1188)
in 9383 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:13:12 INFO TaskSetManager: Finished task 5.0 in stage 260.0 (TID 1190)
in 9383 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 4.0 in stage 260.0 (TID 1189)
in 9384 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 2.0 in stage 260.0 (TID 1187)
in 9384 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 7.0 in stage 262.0 (TID 1201)
in 8705 ms on 10.183.229.123 (executor 0) (1/9)
25/02/06 06:13:13 INFO TaskSetManager: Starting task 6.0 in stage 264.0 (TID 1209)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 7.0 in stage 264.0 (TID 1210)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 8.0 in stage 264.0 (TID 1211)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 0.0 in stage 266.0 (TID 1212)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:13:13 INFO MemoryStore: Block broadcast_85 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 5.0 in stage 258.0 (TID 1181)
in 9539 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:13:13 INFO TaskSchedulerImpl: Removed TaskSet 258.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:13 INFO DAGScheduler: ShuffleMapStage 258 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 132.216 s
25/02/06 06:13:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:13 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 260,
ShuffleMapStage 362, ShuffleMapStage 341, ShuffleMapStage 262)
25/02/06 06:13:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:13 INFO DAGScheduler: failed: Set()
25/02/06 06:13:13 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:13:13 INFO TaskSetManager: Finished task 7.0 in stage 260.0 (TID 1192)
in 9440 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 6.0 in stage 262.0 (TID 1200)
in 8852 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 0.0 in stage 262.0 (TID 1194)
in 9442 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:13:13 INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:13:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:13 INFO TaskSetManager: Finished task 1.0 in stage 260.0 (TID 1186)
in 9675 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:13:13 INFO TaskSchedulerImpl: Removed TaskSet 260.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:13 INFO TaskSetManager: Finished task 8.0 in stage 262.0 (TID 1202)
in 8830 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 5.0 in stage 262.0 (TID 1199)
in 9270 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:13:13 INFO TaskSetManager: Finished task 0.0 in stage 264.0 (TID 1203)
in 8764 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:13:13 INFO DAGScheduler: ShuffleMapStage 260 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 130.457 s
25/02/06 06:13:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:13 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362,
ShuffleMapStage 341, ShuffleMapStage 262)
25/02/06 06:13:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:13 INFO DAGScheduler: failed: Set()
25/02/06 06:13:13 INFO TaskSetManager: Starting task 1.0 in stage 266.0 (TID 1213)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 2.0 in stage 266.0 (TID 1214)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 3.0 in stage 266.0 (TID 1215)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 4.0 in stage 266.0 (TID 1216)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 5.0 in stage 266.0 (TID 1217)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 6.0 in stage 266.0 (TID 1218)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:13:13 INFO TaskSetManager: Starting task 7.0 in stage 266.0 (TID 1219)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:13:13 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:13 INFO BlockManagerInfo: Removed broadcast_79_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:13 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:13 INFO BlockManagerInfo: Removed broadcast_79_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:13 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:13 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:13 INFO SparkContext: Created broadcast 85 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:24 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:13:24 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:13:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:24 INFO AsyncEventQueue: Process of event
SparkListenerTaskEnd(264,0,ShuffleMapTask,Success,org.apache.spark.scheduler.TaskIn
fo@4a177aca,org.apache.spark.executor.ExecutorMetrics@386bcd86,org.apache.spark.exe
cutor.TaskMetrics@26f39f51) by listener DBCEventLoggingListener took 10.914661226s.
25/02/06 06:13:24 INFO TaskSetManager: Finished task 1.0 in stage 262.0 (TID 1195)
in 20586 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:13:24 INFO TaskSetManager: Starting task 8.0 in stage 266.0 (TID 1220)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:13:24 INFO TaskSetManager: Starting task 0.0 in stage 268.0 (TID 1221)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:13:24 INFO TaskSetManager: Starting task 1.0 in stage 268.0 (TID 1222)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:13:24 INFO TaskSetManager: Starting task 2.0 in stage 268.0 (TID 1223)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:13:24 INFO TaskSetManager: Finished task 2.0 in stage 262.0 (TID 1196)
in 20588 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:13:24 INFO TaskSetManager: Finished task 4.0 in stage 262.0 (TID 1198)
in 20588 ms on 10.183.232.45 (executor 2) (8/9)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_79_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_79_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO TaskSetManager: Finished task 3.0 in stage 262.0 (TID 1197)
in 20593 ms on 10.183.232.45 (executor 2) (9/9)
25/02/06 06:13:24 INFO TaskSchedulerImpl: Removed TaskSet 262.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:24 INFO DAGScheduler: ShuffleMapStage 262 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 139.797 s
25/02/06 06:13:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:24 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 264, ShuffleMapStage 266,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:13:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:24 INFO DAGScheduler: failed: Set()
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_82_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_81_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 88 to 10.183.233.191:55538
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_79_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_80_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_82_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_81_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 86 to 10.183.246.72:39896
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 88 to 10.183.246.72:39896
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_82_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_81_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_81_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Removed broadcast_82_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:24 INFO MemoryStore: Block broadcast_86 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 86 to 10.183.229.123:55848
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 88 to 10.183.249.197:36834
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 88 to 10.183.232.45:60256
25/02/06 06:13:24 INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:13:24 INFO SparkContext: Created broadcast 86 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:24 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.232.45:60256
25/02/06 06:13:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:13:37 INFO TaskSetManager: Starting task 3.0 in stage 268.0 (TID 1224)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Finished task 2.0 in stage 264.0 (TID 1205)
in 24512 ms on 10.183.229.123 (executor 0) (2/9)
25/02/06 06:13:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:37 INFO TaskSetManager: Finished task 5.0 in stage 264.0 (TID 1208)
in 24520 ms on 10.183.229.123 (executor 0) (3/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 3.0 in stage 264.0 (TID 1206)
in 24521 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 4.0 in stage 264.0 (TID 1207)
in 24521 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 7.0 in stage 266.0 (TID 1219)
in 24308 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 4.0 in stage 266.0 (TID 1216)
in 24308 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 6.0 in stage 266.0 (TID 1218)
in 24308 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 8.0 in stage 264.0 (TID 1211)
in 24511 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 7.0 in stage 264.0 (TID 1210)
in 24511 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 5.0 in stage 266.0 (TID 1217)
in 24310 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 6.0 in stage 264.0 (TID 1209)
in 24512 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 0.0 in stage 266.0 (TID 1212)
in 24512 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 8.0 in stage 266.0 (TID 1220)
in 13301 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 1.0 in stage 268.0 (TID 1222)
in 13302 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 0.0 in stage 268.0 (TID 1221)
in 13302 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 1.0 in stage 264.0 (TID 1204)
in 33099 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:13:37 INFO TaskSchedulerImpl: Removed TaskSet 264.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:37 INFO TaskSetManager: Finished task 3.0 in stage 266.0 (TID 1215)
in 24314 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 1.0 in stage 266.0 (TID 1213)
in 24315 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:13:37 INFO TaskSetManager: Finished task 2.0 in stage 266.0 (TID 1214)
in 24315 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:13:37 INFO TaskSchedulerImpl: Removed TaskSet 266.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:13:37 INFO DAGScheduler: ShuffleMapStage 264 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 150.589 s
25/02/06 06:13:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 266, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:13:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:37 INFO DAGScheduler: failed: Set()
25/02/06 06:13:37 INFO TaskSetManager: Starting task 4.0 in stage 268.0 (TID 1225)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 5.0 in stage 268.0 (TID 1226)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 6.0 in stage 268.0 (TID 1227)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 7.0 in stage 268.0 (TID 1228)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 8.0 in stage 268.0 (TID 1229)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 0.0 in stage 271.0 (TID 1230)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 1.0 in stage 271.0 (TID 1231)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 2.0 in stage 271.0 (TID 1232)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 3.0 in stage 271.0 (TID 1233)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 4.0 in stage 271.0 (TID 1234)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 5.0 in stage 271.0 (TID 1235)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 0.0 in stage 273.0 (TID 1236)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 1.0 in stage 273.0 (TID 1237)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 2.0 in stage 273.0 (TID 1238)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 3.0 in stage 273.0 (TID 1239)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 4.0 in stage 273.0 (TID 1240)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 5.0 in stage 273.0 (TID 1241)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 6.0 in stage 273.0 (TID 1242)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:13:37 INFO TaskSetManager: Starting task 7.0 in stage 273.0 (TID 1243)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:13:37 INFO DAGScheduler: ShuffleMapStage 266 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 148.569 s
25/02/06 06:13:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:13:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 268,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:13:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:13:37 INFO DAGScheduler: failed: Set()
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.233.191:55538
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.249.197:36834
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.229.123:55848
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:13:37 INFO TaskSetManager: Finished task 2.0 in stage 268.0 (TID 1223)
in 13421 ms on 10.183.232.45 (executor 2) (3/9)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 102 to 10.183.233.191:55538
25/02/06 06:13:37 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(217,1738822296012,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@484e9b3a, org.apache.spark.scheduler.StageInfo@4763d260,
org.apache.spark.scheduler.StageInfo@62cfc476,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 68.832507842s.
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 102 to 10.183.229.123:55848
25/02/06 06:13:37 INFO MemoryStore: Block broadcast_87 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Removed broadcast_83_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 102 to 10.183.249.197:36834
25/02/06 06:13:37 INFO BlockManagerInfo: Removed broadcast_83_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Removed broadcast_83_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Removed broadcast_83_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MemoryStore: Block broadcast_87_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:13:37 INFO SparkContext: Created broadcast 87 from broadcast at
TaskSetManager.scala:711
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 92 to 10.183.233.191:55538
25/02/06 06:13:37 INFO BlockManagerInfo: Removed broadcast_83_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.232.45:60256
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 92 to 10.183.246.72:39896
25/02/06 06:13:37 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 85 to 10.183.246.72:39896
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 92 to 10.183.249.197:36834
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 92 to 10.183.232.45:60256
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 102 to 10.183.246.72:39896
25/02/06 06:13:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 102 to 10.183.232.45:60256
25/02/06 06:13:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:13:50 INFO TaskSetManager: Finished task 5.0 in stage 268.0 (TID 1226)
in 12676 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 4.0 in stage 273.0 (TID 1240)
in 12675 ms on 10.183.233.191 (executor 5) (1/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 0.0 in stage 273.0 (TID 1236)
in 12677 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 7.0 in stage 268.0 (TID 1228)
in 12677 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 6.0 in stage 273.0 (TID 1242)
in 12677 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 2.0 in stage 273.0 (TID 1238)
in 12677 ms on 10.183.246.72 (executor 6) (4/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 3.0 in stage 268.0 (TID 1224)
in 12705 ms on 10.183.229.123 (executor 0) (6/9)
25/02/06 06:13:50 INFO TaskSetManager: Starting task 8.0 in stage 273.0 (TID 1244)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 0.0 in stage 276.0 (TID 1245)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 1.0 in stage 276.0 (TID 1246)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 2.0 in stage 276.0 (TID 1247)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:13:50 INFO BlockManagerInfo: Removed broadcast_84_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:50 INFO BlockManagerInfo: Removed broadcast_84_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 92 to 10.183.233.191:55538
25/02/06 06:13:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 101 to 10.183.233.191:55538
25/02/06 06:13:50 INFO BlockManagerInfo: Removed broadcast_84_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:13:50 INFO BlockManagerInfo: Removed broadcast_85_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:13:50 INFO BlockManagerInfo: Removed broadcast_85_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:13:50 INFO TaskSetManager: Starting task 3.0 in stage 276.0 (TID 1248)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 4.0 in stage 276.0 (TID 1249)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Finished task 7.0 in stage 273.0 (TID 1243)
in 12813 ms on 10.183.232.45 (executor 2) (5/9)
25/02/06 06:13:50 INFO TaskSetManager: Starting task 5.0 in stage 276.0 (TID 1250)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 0.0 in stage 279.0 (TID 1251)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 1.0 in stage 279.0 (TID 1252)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 2.0 in stage 279.0 (TID 1253)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 3.0 in stage 279.0 (TID 1254)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Finished task 3.0 in stage 273.0 (TID 1239)
in 12815 ms on 10.183.232.45 (executor 2) (6/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 1.0 in stage 273.0 (TID 1237)
in 12815 ms on 10.183.249.197 (executor 7) (7/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 8.0 in stage 268.0 (TID 1229)
in 12816 ms on 10.183.232.45 (executor 2) (7/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 6.0 in stage 268.0 (TID 1227)
in 12817 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:13:50 INFO TaskSetManager: Finished task 5.0 in stage 273.0 (TID 1241)
in 12816 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:13:50 INFO TaskSetManager: Starting task 4.0 in stage 279.0 (TID 1255)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 0.0 in stage 282.0 (TID 1256)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:13:50 INFO TaskSetManager: Starting task 1.0 in stage 282.0 (TID 1257)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:13:50 INFO ProgressReporter$: Reporting progress for running commands:
5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9 [186
occurrences]
25/02/06 06:14:02 INFO TaskSetManager: Finished task 4.0 in stage 268.0 (TID 1225)
in 24564 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:14:02 INFO TaskSchedulerImpl: Removed TaskSet 268.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:14:02 INFO DAGScheduler: ShuffleMapStage 268 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 166.988 s
25/02/06 06:14:02 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:14:02 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 329, ShuffleMapStage 308, ShuffleMapStage 410,
ShuffleMapStage 279, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 273, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 271, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 282,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:14:02 INFO DAGScheduler: waiting: Set()
25/02/06 06:14:02 INFO DAGScheduler: failed: Set()
25/02/06 06:14:02 INFO AsyncEventQueue: Process of event
SparkListenerTaskStart(276,0,org.apache.spark.scheduler.TaskInfo@1025cba9) by
listener DBCEventLoggingListener took 11.720266677s.
25/02/06 06:13:37 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097340) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097340)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:14:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 101 to 10.183.246.72:39896
25/02/06 06:14:02 INFO BlockManagerInfo: Removed broadcast_85_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:02 INFO BlockManagerInfo: Removed broadcast_84_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:14:02 INFO BlockManagerInfo: Removed broadcast_85_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 106 to 10.183.249.197:36834
25/02/06 06:14:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 103 to 10.183.249.197:36834
25/02/06 06:14:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 103 to 10.183.232.45:60256
25/02/06 06:14:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:02 INFO CodeGenerator: Code generated in 11877.604927 ms
25/02/06 06:14:02 INFO CodeGenerator: Code generated in 24137.841577 ms
25/02/06 06:14:14 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:14 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:14 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:14 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:26 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:26 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:26 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:26 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:14:37 INFO DAGScheduler: Registering RDD 1512
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 233
25/02/06 06:14:37 INFO DAGScheduler: Got map stage job 236
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:14:37 INFO DAGScheduler: Final stage: ShuffleMapStage 431
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:14:37 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
430)
25/02/06 06:14:37 INFO DAGScheduler: Missing parents: List()
25/02/06 06:14:37 INFO DAGScheduler: Submitting ShuffleMapStage 431
(MapPartitionsRDD[1512] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:14:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 103 to 10.183.229.123:55848
25/02/06 06:14:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 101 to 10.183.229.123:55848
25/02/06 06:14:37 INFO BlockManagerInfo: Removed broadcast_84_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:14:48 INFO BlockManagerInfo: Removed broadcast_85_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:14:48 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:48 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:48 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:48 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:14:58 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:14:58 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:14:58 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:14:58 INFO CodeGenerator: Code generated in 80891.178079 ms
25/02/06 06:14:58 INFO CodeGenerator: Code generated in 80937.025944 ms
25/02/06 06:15:09 INFO DAGScheduler: Registering RDD 1521
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 234
25/02/06 06:15:09 INFO DAGScheduler: Got map stage job 237
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:15:09 INFO DAGScheduler: Final stage: ShuffleMapStage 434
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:15:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
433)
25/02/06 06:15:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:15:09 INFO DAGScheduler: Submitting ShuffleMapStage 434
(MapPartitionsRDD[1521] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:15:31 INFO BlockManagerInfo: Removed broadcast_86_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:15:41 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:15:41 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:15:55 INFO AsyncEventQueue: Process of event
SparkListenerExecutorMetricsUpdate(execId=7, ...) by listener QueryWatchdogListener
took 77.945497184s.
25/02/06 06:15:55 INFO TaskSetManager: Finished task 3.0 in stage 271.0 (TID 1233)
in 137509 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 4.0 in stage 276.0 (TID 1249)
in 124699 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 2.0 in stage 276.0 (TID 1247)
in 124828 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 3.0 in stage 276.0 (TID 1248)
in 124700 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 2.0 in stage 271.0 (TID 1232)
in 137512 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 4.0 in stage 271.0 (TID 1234)
in 137512 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 2.0 in stage 279.0 (TID 1253)
in 124697 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 5.0 in stage 271.0 (TID 1235)
in 137513 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 0.0 in stage 271.0 (TID 1230)
in 137514 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 4.0 in stage 279.0 (TID 1255)
in 124695 ms on 10.183.249.197 (executor 7) (2/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 1.0 in stage 282.0 (TID 1257)
in 124695 ms on 10.183.249.197 (executor 7) (1/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 0.0 in stage 282.0 (TID 1256)
in 124695 ms on 10.183.249.197 (executor 7) (2/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 1.0 in stage 276.0 (TID 1246)
in 124834 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 1.0 in stage 271.0 (TID 1231)
in 137516 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:15:55 INFO TaskSchedulerImpl: Removed TaskSet 271.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:15:55 INFO TaskSetManager: Finished task 3.0 in stage 279.0 (TID 1254)
in 124700 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 8.0 in stage 273.0 (TID 1244)
in 124836 ms on 10.183.233.191 (executor 5) (9/9)
25/02/06 06:15:55 INFO TaskSchedulerImpl: Removed TaskSet 273.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:15:55 INFO TaskSetManager: Finished task 1.0 in stage 279.0 (TID 1252)
in 124701 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:15:55 INFO TaskSetManager: Finished task 0.0 in stage 276.0 (TID 1245)
in 124836 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:15:55 INFO TaskSetManager: Starting task 2.0 in stage 282.0 (TID 1258)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 3.0 in stage 282.0 (TID 1259)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 4.0 in stage 282.0 (TID 1260)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 0.0 in stage 284.0 (TID 1261)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 1.0 in stage 284.0 (TID 1262)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 2.0 in stage 284.0 (TID 1263)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 3.0 in stage 284.0 (TID 1264)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 4.0 in stage 284.0 (TID 1265)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 5.0 in stage 284.0 (TID 1266)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 6.0 in stage 284.0 (TID 1267)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 7.0 in stage 284.0 (TID 1268)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 8.0 in stage 284.0 (TID 1269)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 0.0 in stage 286.0 (TID 1270)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 1.0 in stage 286.0 (TID 1271)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 2.0 in stage 286.0 (TID 1272)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 3.0 in stage 286.0 (TID 1273)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 4.0 in stage 286.0 (TID 1274)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:15:55 INFO TaskSetManager: Starting task 5.0 in stage 286.0 (TID 1275)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:16:07 INFO MemoryStore: Block broadcast_88 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:16:07 INFO MemoryStore: Block broadcast_89 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:16:07 INFO MemoryStore: Block broadcast_89_piece0 stored as bytes in
memory (estimated size 38.8 KiB, free 20.2 GiB)
25/02/06 06:16:07 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.232.44:37705 (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:16:07 INFO SparkContext: Created broadcast 89 from broadcast at
TaskSetManager.scala:711
25/02/06 06:16:07 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 106 to 10.183.246.72:39896
25/02/06 06:16:07 INFO DAGScheduler: Registering RDD 1522
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 235
25/02/06 06:16:07 INFO DAGScheduler: Got map stage job 238
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:16:07 INFO DAGScheduler: Final stage: ShuffleMapStage 437
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:16:07 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
436)
25/02/06 06:16:07 INFO DAGScheduler: Missing parents: List()
25/02/06 06:16:07 INFO DAGScheduler: Submitting ShuffleMapStage 437
(MapPartitionsRDD[1522] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:16:07 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 431 (MapPartitionsRDD[1512] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:16:07 INFO TaskSchedulerImpl: Adding task set 431.0 with 5 tasks
resource profile 0
25/02/06 06:16:07 INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:16:07 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:16:07 INFO SparkContext: Created broadcast 88 from broadcast at
TaskSetManager.scala:711
25/02/06 06:16:07 INFO BlockManagerInfo: Removed broadcast_87_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:07 INFO BlockManagerInfo: Removed broadcast_87_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:07 INFO BlockManagerInfo: Removed broadcast_87_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:07 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:07 WARN DriverDaemon$$anon$1: Could not send message because of
exception message: Futures timed out after [30 seconds], retry attempts: 1/30,
message transformed due to serialization exception: false
java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)
at scala.concurrent.Await$.$anonfun$result$1(package.scala:223)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)
at scala.concurrent.Await$.result(package.scala:146)
at
com.databricks.spark.chauffeur.common.messagewriter.MessageWriterImpl.sendMessage(M
essageWriterImpl.scala:192)
at com.databricks.spark.chauffeur.common.messagewriter.MessageWriterImpl.
$anonfun$waitAndProcessMessage$1(MessageWriterImpl.scala:127)
at
com.databricks.spark.chauffeur.common.messagewriter.MessageWriterImpl.com$databrick
s$spark$chauffeur$common$messagewriter$MessageWriterImpl$
$waitAndProcessMessage(MessageWriterImpl.scala:133)
at com.databricks.spark.chauffeur.common.messagewriter.MessageWriterImpl$
$anon$1.$anonfun$run$1(MessageWriterImpl.scala:84)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.spark.chauffeur.common.messagewriter.MessageWriterImpl$
$anon$1.run(MessageWriterImpl.scala:84)
at java.lang.Thread.run(Thread.java:750)
25/02/06 06:16:07 INFO TaskSetManager: TaskSet 431.0 using PreferredLocationsV1
25/02/06 06:16:07 INFO FairSchedulableBuilder: Added task set TaskSet_431.0 tasks
to pool 5400065285585087590
25/02/06 06:16:07 INFO DAGScheduler: ShuffleMapStage 271
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
290.077 s
25/02/06 06:16:07 INFO TaskSetManager: Starting task 6.0 in stage 286.0 (TID 1276)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:16:07 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:07 INFO TaskSetManager: Starting task 7.0 in stage 286.0 (TID 1277)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:16:07 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 302,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 273,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 300, ShuffleMapStage 292, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 276, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:07 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:07 INFO DAGScheduler: failed: Set()
25/02/06 06:16:07 INFO TaskSetManager: Finished task 5.0 in stage 276.0 (TID 1250)
in 136805 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:16:07 INFO TaskSchedulerImpl: Removed TaskSet 276.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:07 INFO TaskSetManager: Finished task 0.0 in stage 279.0 (TID 1251)
in 136811 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:16:07 INFO TaskSchedulerImpl: Removed TaskSet 279.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:07 INFO DataSourceFactory$: DataSource Jdbc URL:
jdbc:mariadb://md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com:3306/
organization8517174019523704?
useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertif
icate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt
25/02/06 06:16:07 INFO HikariDataSource: metastore-monitor - Starting...
25/02/06 06:16:07 INFO BlockManagerInfo: Removed broadcast_87_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:07 INFO HikariDataSource: metastore-monitor - Start completed.
25/02/06 06:16:19 INFO BlockManagerInfo: Removed broadcast_87_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:16:19 WARN ReliableJettyClient: Delaying RPC
com.databricks.backend.daemon.data.common.DataMessages$GetMountsV2 since peer
BoundRPCClient[http://10.183.251.100:7070 with NoProxy] is
DOWN(java.io.IOException: java.util.concurrent.TimeoutException: Timed out after 10
seconds)
25/02/06 06:16:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:19 INFO DAGScheduler: ShuffleMapStage 273 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 301.778 s
25/02/06 06:16:19 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:19 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 302,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ShuffleMapStage 292, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 276,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:19 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:19 INFO DAGScheduler: failed: Set()
25/02/06 06:16:20 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:20 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.249.197:40023 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:20 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:20 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:20 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.229.123:38441 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 87 to 10.183.232.45:60256
25/02/06 06:16:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 87 to 10.183.249.197:36834
25/02/06 06:16:20 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 89 to 10.183.249.197:36834
25/02/06 06:16:31 INFO DAGScheduler: Registering RDD 1524
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 236
25/02/06 06:16:31 INFO DAGScheduler: Got map stage job 239
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:16:31 INFO DAGScheduler: Final stage: ShuffleMapStage 440
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:16:31 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
439)
25/02/06 06:16:31 INFO DAGScheduler: Missing parents: List()
25/02/06 06:16:31 INFO DAGScheduler: Submitting ShuffleMapStage 440
(MapPartitionsRDD[1524] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:16:31 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 434 (MapPartitionsRDD[1521] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:16:31 INFO TaskSchedulerImpl: Adding task set 434.0 with 6 tasks
resource profile 0
25/02/06 06:16:31 INFO TaskSetManager: TaskSet 434.0 using PreferredLocationsV1
25/02/06 06:16:31 INFO FairSchedulableBuilder: Added task set TaskSet_434.0 tasks
to pool 5400065285585087590
25/02/06 06:16:31 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 437 (MapPartitionsRDD[1522] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:16:31 INFO TaskSchedulerImpl: Adding task set 437.0 with 6 tasks
resource profile 0
25/02/06 06:16:31 INFO TaskSetManager: TaskSet 437.0 using PreferredLocationsV1
25/02/06 06:16:31 INFO FairSchedulableBuilder: Added task set TaskSet_437.0 tasks
to pool 5400065285585087590
25/02/06 06:16:31 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.233.191:42923 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:31 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:31 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.232.45:45579 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:31 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 89 to 10.183.229.123:55848
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 87 to 10.183.229.123:55848
25/02/06 06:16:31 INFO DAGScheduler: ShuffleMapStage 276
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
312.365 s
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 89 to 10.183.232.45:60256
25/02/06 06:16:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:31 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 279, ShuffleMapStage 302,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300,
ShuffleMapStage 292, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 294, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:31 INFO DAGScheduler: failed: Set()
25/02/06 06:16:31 WARN HikariPool: HikariPool-2 - Thread starvation or clock leap
detected (housekeeper delta=1m54s542ms906µs836ns).
25/02/06 06:16:31 WARN HikariPool: HikariPool-1 - Thread starvation or clock leap
detected (housekeeper delta=1m54s543ms48µs15ns).
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:16:31 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:16:31 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:16:31 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:16:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 87 to 10.183.233.191:55538
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 89 to 10.183.233.191:55538
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 87 to 10.183.246.72:39896
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:16:31 INFO HikariDataSource: metastore-monitor - Shutdown initiated...
25/02/06 06:16:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:31 INFO TaskSetManager: Starting task 8.0 in stage 286.0 (TID 1278)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:16:31 INFO TaskSetManager: Finished task 3.0 in stage 282.0 (TID 1259)
in 36728 ms on 10.183.246.72 (executor 6) (3/5)
25/02/06 06:16:31 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on
10.183.246.72:43231 (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 89 to 10.183.246.72:39896
25/02/06 06:16:31 INFO TaskSetManager: Starting task 0.0 in stage 289.0 (TID 1279)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:16:31 INFO TaskSetManager: Finished task 4.0 in stage 282.0 (TID 1260)
in 36756 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 107 to 10.183.246.72:39896
25/02/06 06:16:31 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:16:31 INFO TaskSetManager: Starting task 1.0 in stage 289.0 (TID 1280)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:16:31 INFO TaskSetManager: Finished task 2.0 in stage 282.0 (TID 1258)
in 36779 ms on 10.183.246.72 (executor 6) (5/5)
25/02/06 06:16:31 INFO TaskSchedulerImpl: Removed TaskSet 282.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:31 INFO DAGScheduler: ShuffleMapStage 279
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
312.592 s
25/02/06 06:16:31 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:31 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 282, ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:31 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:31 INFO DAGScheduler: failed: Set()
25/02/06 06:16:31 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 440 (MapPartitionsRDD[1524] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:16:45 INFO TaskSchedulerImpl: Adding task set 440.0 with 6 tasks
resource profile 0
25/02/06 06:16:45 WARN ReliableJettyClient: Delaying RPC
com.databricks.backend.daemon.data.common.DataMessages$GetMountsV2 since peer
BoundRPCClient[http://10.183.251.100:7070 with NoProxy] is
DOWN(java.io.IOException: java.util.concurrent.TimeoutException: Timed out after 10
seconds)
25/02/06 06:16:45 INFO TaskSetManager: TaskSet 440.0 using PreferredLocationsV1
25/02/06 06:16:45 INFO FairSchedulableBuilder: Added task set TaskSet_440.0 tasks
to pool 5400065285585087590
25/02/06 06:16:45 INFO DAGScheduler: ShuffleMapStage 282
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
324.124 s
25/02/06 06:16:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 305, ShuffleMapStage 284, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 286, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:16:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:45 INFO DAGScheduler: failed: Set()
25/02/06 06:16:45 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:45 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:45 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:45 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:45 INFO TaskSetManager: Starting task 2.0 in stage 289.0 (TID 1281)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 3.0 in stage 289.0 (TID 1282)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 4.0 in stage 289.0 (TID 1283)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Finished task 1.0 in stage 286.0 (TID 1271)
in 50108 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 2.0 in stage 284.0 (TID 1263)
in 50110 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 4.0 in stage 286.0 (TID 1274)
in 50110 ms on 10.183.232.45 (executor 2) (2/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 5.0 in stage 284.0 (TID 1266)
in 50112 ms on 10.183.249.197 (executor 7) (2/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 3.0 in stage 286.0 (TID 1273)
in 50111 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 0.0 in stage 286.0 (TID 1270)
in 50112 ms on 10.183.249.197 (executor 7) (4/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 1.0 in stage 284.0 (TID 1262)
in 50113 ms on 10.183.249.197 (executor 7) (3/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 6.0 in stage 284.0 (TID 1267)
in 50120 ms on 10.183.232.45 (executor 2) (4/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 1.0 in stage 289.0 (TID 1280)
in 13344 ms on 10.183.246.72 (executor 6) (1/5)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 8.0 in stage 286.0 (TID 1278)
in 13396 ms on 10.183.246.72 (executor 6) (5/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 0.0 in stage 289.0 (TID 1279)
in 13369 ms on 10.183.246.72 (executor 6) (2/5)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 8.0 in stage 284.0 (TID 1269)
in 50122 ms on 10.183.229.123 (executor 0) (5/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 0.0 in stage 284.0 (TID 1261)
in 50126 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:16:45 INFO TaskSetManager: Starting task 0.0 in stage 292.0 (TID 1284)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 1.0 in stage 292.0 (TID 1285)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 2.0 in stage 292.0 (TID 1286)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 3.0 in stage 292.0 (TID 1287)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 4.0 in stage 292.0 (TID 1288)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 5.0 in stage 292.0 (TID 1289)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 0.0 in stage 294.0 (TID 1290)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 1.0 in stage 294.0 (TID 1291)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 2.0 in stage 294.0 (TID 1292)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 3.0 in stage 294.0 (TID 1293)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:16:45 INFO MemoryStore: Block broadcast_90 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 108 to 10.183.246.72:39896
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 108 to 10.183.229.123:55848
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 107 to 10.183.232.45:60256
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 108 to 10.183.232.45:60256
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 108 to 10.183.249.197:36834
25/02/06 06:16:45 INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:16:45 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:16:45 INFO SparkContext: Created broadcast 90 from broadcast at
TaskSetManager.scala:711
25/02/06 06:16:45 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:45 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on
10.183.249.197:40023 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 91 to 10.183.246.72:39896
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 91 to 10.183.249.197:36834
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:16:45 INFO TaskSetManager: Starting task 4.0 in stage 294.0 (TID 1294)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 5.0 in stage 294.0 (TID 1295)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 6.0 in stage 294.0 (TID 1296)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 7.0 in stage 294.0 (TID 1297)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 8.0 in stage 294.0 (TID 1298)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 0.0 in stage 297.0 (TID 1299)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Starting task 1.0 in stage 297.0 (TID 1300)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:16:45 INFO TaskSetManager: Finished task 7.0 in stage 284.0 (TID 1268)
in 50253 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 3.0 in stage 284.0 (TID 1264)
in 50254 ms on 10.183.233.191 (executor 5) (8/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 2.0 in stage 286.0 (TID 1272)
in 50253 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:16:45 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 5.0 in stage 286.0 (TID 1275)
in 50254 ms on 10.183.233.191 (executor 5) (7/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 7.0 in stage 286.0 (TID 1277)
in 38172 ms on 10.183.229.123 (executor 0) (8/9)
25/02/06 06:16:45 INFO TaskSetManager: Finished task 4.0 in stage 284.0 (TID 1265)
in 50260 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:16:45 INFO TaskSchedulerImpl: Removed TaskSet 284.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:45 INFO TaskSetManager: Finished task 6.0 in stage 286.0 (TID 1276)
in 38175 ms on 10.183.229.123 (executor 0) (9/9)
25/02/06 06:16:45 INFO TaskSchedulerImpl: Removed TaskSet 286.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:45 INFO DAGScheduler: ShuffleMapStage 284 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 323.179 s
25/02/06 06:16:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 91 to 10.183.233.191:55538
25/02/06 06:16:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 305, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 286,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 297, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:45 INFO DAGScheduler: failed: Set()
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:16:45 INFO TaskSetManager: Starting task 2.0 in stage 297.0 (TID 1301)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:16:45 INFO DAGScheduler: ShuffleMapStage 286 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 323.209 s
25/02/06 06:16:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:45 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 305, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 289, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:45 INFO DAGScheduler: failed: Set()
25/02/06 06:16:45 INFO TaskSetManager: Finished task 3.0 in stage 292.0 (TID 1287)
in 149 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:16:45 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 99 to 10.183.229.123:55848
25/02/06 06:16:45 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:57 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:57 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:57 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:57 INFO TaskSetManager: Starting task 3.0 in stage 297.0 (TID 1302)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Starting task 4.0 in stage 297.0 (TID 1303)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Starting task 5.0 in stage 297.0 (TID 1304)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Starting task 0.0 in stage 300.0 (TID 1305)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Starting task 1.0 in stage 300.0 (TID 1306)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Finished task 1.0 in stage 294.0 (TID 1291)
in 12562 ms on 10.183.246.72 (executor 6) (1/9)
25/02/06 06:16:57 INFO TaskSetManager: Finished task 7.0 in stage 294.0 (TID 1297)
in 12508 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:16:57 INFO TaskSetManager: Finished task 5.0 in stage 292.0 (TID 1289)
in 12579 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:16:57 INFO TaskSetManager: Finished task 4.0 in stage 294.0 (TID 1294)
in 12509 ms on 10.183.233.191 (executor 5) (3/9)
25/02/06 06:16:57 INFO TaskSetManager: Starting task 2.0 in stage 300.0 (TID 1307)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Starting task 3.0 in stage 300.0 (TID 1308)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Finished task 6.0 in stage 294.0 (TID 1296)
in 12510 ms on 10.183.233.191 (executor 5) (4/9)
25/02/06 06:16:57 INFO TaskSetManager: Finished task 5.0 in stage 294.0 (TID 1295)
in 12510 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.249.197:36834
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 99 to 10.183.246.72:39896
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 91 to 10.183.229.123:55848
25/02/06 06:16:57 INFO TaskSetManager: Finished task 0.0 in stage 292.0 (TID 1284)
in 12590 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:16:57 INFO TaskSetManager: Starting task 4.0 in stage 300.0 (TID 1309)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:16:57 INFO TaskSetManager: Finished task 3.0 in stage 294.0 (TID 1293)
in 12579 ms on 10.183.246.72 (executor 6) (6/9)
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 98 to 10.183.233.191:55538
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 99 to 10.183.233.191:55538
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 98 to 10.183.246.72:39896
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO AsyncEventQueue: Process of event
SparkListenerJobEnd(187,1738822605389,JobSucceeded) by listener
DBCEventLoggingListener took 12.416930795s.
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 98 to 10.183.229.123:55848
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.232.45:45579 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:57 WARN DatabricksInstrumentation:V3: Detected 1 slow requests, the
latest request was 'List
databricks-workspace-stack-f2c65-bucket:/ireland-prod/8517174019523704/' that
returned 0 objects within 12667.748 ms (0.000 obj/ms). This might be caused by
bucket versioning, e.g. a number of versions for the key, bucket region being
different from cluster workers, or AWS network issues.
25/02/06 06:16:57 INFO DriverCorral: DBFS health check ok
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.246.72:43231 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:57 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.233.191:42923 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 3.0 in stage 300.0 (TID 1308)
in 244 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 5.0 in stage 300.0 (TID 1310)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Finished task 5.0 in stage 297.0 (TID 1304)
in 267 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 0.0 in stage 300.0 (TID 1305)
in 267 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 3.0 in stage 297.0 (TID 1302)
in 268 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 0.0 in stage 302.0 (TID 1311)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 1.0 in stage 302.0 (TID 1312)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 2.0 in stage 302.0 (TID 1313)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 3.0 in stage 302.0 (TID 1314)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Finished task 2.0 in stage 297.0 (TID 1301)
in 12706 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 1.0 in stage 300.0 (TID 1306)
in 277 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 4.0 in stage 302.0 (TID 1315)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Finished task 2.0 in stage 300.0 (TID 1307)
in 278 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 5.0 in stage 302.0 (TID 1316)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:16:58 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.229.123:38441 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 8.0 in stage 294.0 (TID 1298)
in 12799 ms on 10.183.229.123 (executor 0) (7/9)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 6.0 in stage 302.0 (TID 1317)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:16:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:16:58 INFO TaskSetManager: Finished task 4.0 in stage 297.0 (TID 1303)
in 312 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 7.0 in stage 302.0 (TID 1318)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:16:58 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:16:58 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.249.197:40023 in memory (size: 38.8 KiB, free: 18.4 GiB)
25/02/06 06:16:58 INFO TaskSetManager: Starting task 8.0 in stage 302.0 (TID 1319)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 0.0 in stage 305.0 (TID 1320)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Finished task 2.0 in stage 289.0 (TID 1281)
in 13051 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 3.0 in stage 289.0 (TID 1282)
in 13051 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 1.0 in stage 292.0 (TID 1285)
in 12981 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 5.0 in stage 300.0 (TID 1310)
in 138 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 4.0 in stage 292.0 (TID 1288)
in 12982 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:16:58 INFO TaskSetManager: Finished task 4.0 in stage 289.0 (TID 1283)
in 13052 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:16:58 INFO TaskSchedulerImpl: Removed TaskSet 289.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:58 INFO TaskSetManager: Finished task 2.0 in stage 292.0 (TID 1286)
in 12983 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:16:58 INFO TaskSchedulerImpl: Removed TaskSet 292.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:16:58 INFO TaskSetManager: Starting task 1.0 in stage 305.0 (TID 1321)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 2.0 in stage 305.0 (TID 1322)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 3.0 in stage 305.0 (TID 1323)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 4.0 in stage 305.0 (TID 1324)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:16:58 INFO TaskSetManager: Starting task 0.0 in stage 308.0 (TID 1325)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:16:58 INFO BlockManagerInfo: Removed broadcast_88_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:16:58 INFO DAGScheduler: ShuffleMapStage 289
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
336.036 s
25/02/06 06:16:58 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:58 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 292,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 305, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:16:58 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:58 INFO DAGScheduler: failed: Set()
25/02/06 06:16:58 INFO DAGScheduler: ShuffleMapStage 292
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
336.032 s
25/02/06 06:16:58 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:16:58 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 311,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 305,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 297, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:16:58 INFO DAGScheduler: waiting: Set()
25/02/06 06:16:58 INFO DAGScheduler: failed: Set()
25/02/06 06:17:11 INFO TaskSetManager: Finished task 1.0 in stage 297.0 (TID 1300)
in 26213 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:17:11 INFO TaskSetManager: Finished task 0.0 in stage 297.0 (TID 1299)
in 26214 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:11 INFO TaskSchedulerImpl: Removed TaskSet 297.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:11 INFO DAGScheduler: ShuffleMapStage 297
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
347.626 s
25/02/06 06:17:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:11 INFO TaskSetManager: Starting task 1.0 in stage 308.0 (TID 1326)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:17:11 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 294, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 311,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 305,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:11 INFO DAGScheduler: failed: Set()
25/02/06 06:17:11 INFO TaskSetManager: Starting task 2.0 in stage 308.0 (TID 1327)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:11 INFO HikariDataSource: metastore-monitor - Shutdown completed.
25/02/06 06:17:11 WARN MetastoreMonitor: Failed to connect to the metastore
InternalMysqlMetastore(DbMetastoreConfig{host=md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-
1.rds.amazonaws.com, port=3306, dbName=organization8517174019523704,
user=[REDACTED]}). (timeSinceLastSuccess=370444)
org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException:
java.sql.SQLTransientConnectionException: metastore-monitor - Connection is not
available, request timed out after 24498ms.
at org.skife.jdbi.v2.DBI.open(DBI.java:230)
at com.databricks.backend.daemon.driver.MetastoreMonitor.
$anonfun$checkMetastore$1(MetastoreMonitor.scala:195)
at com.databricks.backend.daemon.driver.MetastoreMonitor.
$anonfun$checkMetastore$1$adapted(MetastoreMonitor.scala:194)
at
com.databricks.common.database.DatabaseUtils$.withDBI(DatabaseUtils.scala:348)
at
com.databricks.backend.daemon.driver.MetastoreMonitor.checkMetastore(MetastoreMonit
or.scala:194)
at com.databricks.backend.daemon.driver.MetastoreMonitor.
$anonfun$doMonitor$1(MetastoreMonitor.scala:158)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: java.sql.SQLTransientConnectionException: metastore-monitor - Connection
is not available, request timed out after 24498ms.
at
com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
at
com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at
com.databricks.common.database.PassThroughHikariDataSource.getConnection(HikariData
SourceTrait.scala:57)
at
com.databricks.common.database.SecretChangeAwareDataSource.getConnection(Standalone
DataSource.scala:67)
at
com.databricks.common.database.MultiDatabaseDataSource.getConnection(MultiDatabaseD
ataSource.scala:51)
at
org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFa
ctory.java:36)
at org.skife.jdbi.v2.DBI.open(DBI.java:212)
... 38 more
25/02/06 06:17:11 INFO TaskSetManager: Finished task 0.0 in stage 294.0 (TID 1290)
in 26294 ms on 10.183.249.197 (executor 7) (8/9)
25/02/06 06:17:11 INFO TaskSetManager: Finished task 2.0 in stage 294.0 (TID 1292)
in 26294 ms on 10.183.249.197 (executor 7) (9/9)
25/02/06 06:17:11 INFO TaskSchedulerImpl: Removed TaskSet 294.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:11 INFO TaskSetManager: Starting task 3.0 in stage 308.0 (TID 1328)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:17:11 INFO TaskSetManager: Starting task 4.0 in stage 308.0 (TID 1329)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:11 INFO DAGScheduler: ShuffleMapStage 294 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 347.679 s
25/02/06 06:17:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:11 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 300, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 305, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:11 INFO DAGScheduler: failed: Set()
25/02/06 06:17:11 INFO BlockManagerInfo: Removed broadcast_89_piece0 on
10.183.232.44:37705 in memory (size: 38.8 KiB, free: 20.2 GiB)
25/02/06 06:17:11 INFO AsyncEventQueue: Process of event
SparkListenerTaskStart(305,0,org.apache.spark.scheduler.TaskInfo@6195324d) by
listener DBCEventLoggingListener took 13.286828773s.
25/02/06 06:17:11 INFO MemoryStore: Block broadcast_91 stored as values in memory
(estimated size 139.3 KiB, free 20.2 GiB)
25/02/06 06:17:11 INFO MemoryStore: Block broadcast_91_piece0 stored as bytes in
memory (estimated size 38.7 KiB, free 20.2 GiB)
25/02/06 06:17:11 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on
10.183.232.44:37705 (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:17:11 INFO SparkContext: Created broadcast 91 from broadcast at
TaskSetManager.scala:711
25/02/06 06:17:11 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on
10.183.233.191:42923 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 95 to 10.183.232.45:60256
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 95 to 10.183.233.191:55538
25/02/06 06:17:11 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on
10.183.232.45:45579 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 95 to 10.183.249.197:36834
25/02/06 06:17:11 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on
10.183.229.123:38441 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 93 to 10.183.249.197:36834
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 90 to 10.183.229.123:55848
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 90 to 10.183.232.45:60256
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 93 to 10.183.229.123:55848
25/02/06 06:17:11 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on
10.183.246.72:43231 (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.232.45:60256
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.229.123:55848
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 90 to 10.183.233.191:55538
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 90 to 10.183.246.72:39896
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.233.191:55538
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 8 to 10.183.246.72:39896
25/02/06 06:17:11 INFO TaskSetManager: Starting task 5.0 in stage 308.0 (TID 1330)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:11 INFO TaskSetManager: Finished task 4.0 in stage 300.0 (TID 1309)
in 14093 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:11 INFO TaskSchedulerImpl: Removed TaskSet 300.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:11 INFO DAGScheduler: ShuffleMapStage 300
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
347.976 s
25/02/06 06:17:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:11 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 305, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:11 INFO DAGScheduler: failed: Set()
25/02/06 06:17:11 ERROR RetryingHMSHandler: Retrying HMSHandler after 2000 ms
(attempt 1 of 10) with error: javax.jdo.JDODataStoreException: HikariPool-1 -
Connection is not available, request timed out after 39889ms.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(Nucleus
JDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
NestedThrowablesStackTrace:
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not
available, request timed out after 39889ms.
at
com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
at
org.datanucleus.store.rdbms.ConnectionProviderPriorityList.getConnection(Connection
ProviderPriorityList.java:57)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnecti
on(ConnectionFactoryImpl.java:402)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getXAResour
ce(ConnectionFactoryImpl.java:361)
at
org.datanucleus.store.connection.ConnectionManagerImpl.allocateConnection(Connectio
nManagerImpl.java:316)
at
org.datanucleus.store.connection.AbstractConnectionFactory.getConnection(AbstractCo
nnectionFactory.java:84)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
347)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
310)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:591)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1855)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
25/02/06 06:17:11 INFO TaskSetManager: Starting task 0.0 in stage 311.0 (TID 1331)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:17:11 INFO TaskSetManager: Finished task 3.0 in stage 308.0 (TID 1328)
in 422 ms on 10.183.249.197 (executor 7) (1/6)
25/02/06 06:17:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 105 to 10.183.249.197:36834
25/02/06 06:17:11 INFO TaskSetManager: Finished task 4.0 in stage 305.0 (TID 1324)
in 13767 ms on 10.183.249.197 (executor 7) (1/5)
25/02/06 06:17:11 INFO TaskSetManager: Starting task 1.0 in stage 311.0 (TID 1332)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Finished task 4.0 in stage 308.0 (TID 1329)
in 481 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:17:12 INFO TaskSetManager: Starting task 2.0 in stage 311.0 (TID 1333)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Finished task 2.0 in stage 305.0 (TID 1322)
in 13820 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:17:12 INFO TaskSetManager: Starting task 3.0 in stage 311.0 (TID 1334)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:17:12 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 105 to 10.183.232.45:60256
25/02/06 06:17:12 INFO TaskSetManager: Starting task 4.0 in stage 311.0 (TID 1335)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Finished task 8.0 in stage 302.0 (TID 1319)
in 13846 ms on 10.183.232.45 (executor 2) (1/9)
25/02/06 06:17:12 INFO TaskSetManager: Finished task 5.0 in stage 308.0 (TID 1330)
in 141 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:17:12 INFO TaskSetManager: Starting task 5.0 in stage 311.0 (TID 1336)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Finished task 3.0 in stage 305.0 (TID 1323)
in 13843 ms on 10.183.233.191 (executor 5) (3/5)
25/02/06 06:17:12 INFO TaskSetManager: Finished task 0.0 in stage 305.0 (TID 1320)
in 13849 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:17:12 INFO TaskSetManager: Starting task 0.0 in stage 314.0 (TID 1337)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Starting task 1.0 in stage 314.0 (TID 1338)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:12 INFO TaskSetManager: Finished task 2.0 in stage 308.0 (TID 1327)
in 547 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:17:12 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 105 to 10.183.229.123:55848
25/02/06 06:17:12 INFO TaskSetManager: Starting task 2.0 in stage 314.0 (TID 1339)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:17:24 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:17:24 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:17:24 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:17:24 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:17:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 96 to 10.183.232.45:60256
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 96 to 10.183.233.191:55538
25/02/06 06:17:24 INFO BlockManagerInfo: Removed broadcast_90_piece0 on
10.183.249.197:40023 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:24 INFO BlockManagerInfo: Removed broadcast_90_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:24 INFO BlockManagerInfo: Removed broadcast_90_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 96 to 10.183.229.123:55848
25/02/06 06:17:24 INFO TaskSetManager: Finished task 1.0 in stage 305.0 (TID 1321)
in 26758 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:17:24 INFO TaskSchedulerImpl: Removed TaskSet 305.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:24 INFO DAGScheduler: ShuffleMapStage 305
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
359.880 s
25/02/06 06:17:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:24 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 308,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 323, ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:17:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:24 INFO DAGScheduler: failed: Set()
25/02/06 06:17:24 INFO TaskSetManager: Starting task 3.0 in stage 314.0 (TID 1340)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 4.0 in stage 314.0 (TID 1341)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 5.0 in stage 314.0 (TID 1342)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 0.0 in stage 317.0 (TID 1343)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 1.0 in stage 317.0 (TID 1344)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 2.0 in stage 317.0 (TID 1345)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 3.0 in stage 317.0 (TID 1346)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 4.0 in stage 317.0 (TID 1347)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 5.0 in stage 317.0 (TID 1348)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 0.0 in stage 320.0 (TID 1349)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Starting task 1.0 in stage 320.0 (TID 1350)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:24 INFO TaskSetManager: Finished task 0.0 in stage 308.0 (TID 1325)
in 26773 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:17:24 INFO TaskSetManager: Finished task 4.0 in stage 302.0 (TID 1315)
in 26903 ms on 10.183.233.191 (executor 5) (2/9)
25/02/06 06:17:24 INFO TaskSetManager: Finished task 0.0 in stage 302.0 (TID 1311)
in 26912 ms on 10.183.246.72 (executor 6) (3/9)
25/02/06 06:17:24 INFO TaskSetManager: Finished task 2.0 in stage 311.0 (TID 1333)
in 12966 ms on 10.183.249.197 (executor 7) (1/6)
25/02/06 06:17:24 INFO TaskSetManager: Finished task 0.0 in stage 311.0 (TID 1331)
in 13028 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 96 to 10.183.249.197:36834
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 104 to 10.183.249.197:36834
25/02/06 06:17:24 INFO TaskSetManager: Finished task 1.0 in stage 311.0 (TID 1332)
in 13007 ms on 10.183.249.197 (executor 7) (3/6)
25/02/06 06:17:24 INFO TaskSetManager: Finished task 6.0 in stage 302.0 (TID 1317)
in 26889 ms on 10.183.229.123 (executor 0) (4/9)
25/02/06 06:17:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 104 to 10.183.232.45:60256
25/02/06 06:17:24 INFO BlockManagerInfo: Removed broadcast_90_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 7.0 in stage 302.0 (TID 1318)
in 26883 ms on 10.183.233.191 (executor 5) (5/9)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 5.0 in stage 302.0 (TID 1316)
in 26914 ms on 10.183.233.191 (executor 5) (6/9)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 1.0 in stage 308.0 (TID 1326)
in 13487 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:25 INFO TaskSchedulerImpl: Removed TaskSet 308.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:25 INFO TaskSetManager: Finished task 2.0 in stage 302.0 (TID 1313)
in 26928 ms on 10.183.246.72 (executor 6) (7/9)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 1.0 in stage 302.0 (TID 1312)
in 26929 ms on 10.183.246.72 (executor 6) (8/9)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 3.0 in stage 302.0 (TID 1314)
in 26929 ms on 10.183.246.72 (executor 6) (9/9)
25/02/06 06:17:25 INFO TaskSchedulerImpl: Removed TaskSet 302.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:25 INFO TaskSetManager: Starting task 2.0 in stage 320.0 (TID 1351)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 3.0 in stage 320.0 (TID 1352)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 4.0 in stage 320.0 (TID 1353)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 94 to 10.183.233.191:55538
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 104 to 10.183.233.191:55538
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 104 to 10.183.246.72:39896
25/02/06 06:17:25 INFO BlockManagerInfo: Removed broadcast_90_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:17:25 INFO DAGScheduler: ShuffleMapStage 308
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
359.915 s
25/02/06 06:17:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:25 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 302, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323,
ShuffleMapStage 338, ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 311, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:25 INFO DAGScheduler: failed: Set()
25/02/06 06:17:25 INFO DAGScheduler: ShuffleMapStage 302 (mapPartitionsInternal at
PhotonExec.scala:483) finished in 359.963 s
25/02/06 06:17:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:25 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 311,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326,
ShuffleMapStage 428, ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 314, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:25 INFO DAGScheduler: failed: Set()
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 94 to 10.183.229.123:55848
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 104 to 10.183.229.123:55848
25/02/06 06:17:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:25 INFO TaskSetManager: Starting task 5.0 in stage 320.0 (TID 1354)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Finished task 0.0 in stage 314.0 (TID 1337)
in 13081 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 0.0 in stage 320.0 (TID 1349)
in 170 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:17:25 INFO TaskSetManager: Starting task 0.0 in stage 323.0 (TID 1355)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 94 to 10.183.246.72:39896
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 97 to 10.183.233.191:55538
25/02/06 06:17:25 INFO TaskSetManager: Finished task 5.0 in stage 311.0 (TID 1336)
in 13184 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 5.0 in stage 320.0 (TID 1354)
in 100 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 1.0 in stage 314.0 (TID 1338)
in 13181 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 2.0 in stage 317.0 (TID 1345)
in 265 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 1.0 in stage 317.0 (TID 1344)
in 267 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 3.0 in stage 311.0 (TID 1334)
in 13209 ms on 10.183.232.45 (executor 2) (5/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 5.0 in stage 314.0 (TID 1342)
in 269 ms on 10.183.249.197 (executor 7) (3/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 5.0 in stage 317.0 (TID 1348)
in 266 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 4.0 in stage 314.0 (TID 1341)
in 270 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 1.0 in stage 320.0 (TID 1350)
in 267 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:17:25 INFO TaskSetManager: Starting task 1.0 in stage 323.0 (TID 1356)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 2.0 in stage 323.0 (TID 1357)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 3.0 in stage 323.0 (TID 1358)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 4.0 in stage 323.0 (TID 1359)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 0.0 in stage 326.0 (TID 1360)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 1.0 in stage 326.0 (TID 1361)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 2.0 in stage 326.0 (TID 1362)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 3.0 in stage 326.0 (TID 1363)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 4.0 in stage 326.0 (TID 1364)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 5.0 in stage 326.0 (TID 1365)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 0.0 in stage 329.0 (TID 1366)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 1.0 in stage 329.0 (TID 1367)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Finished task 4.0 in stage 311.0 (TID 1335)
in 13192 ms on 10.183.232.45 (executor 2) (6/6)
25/02/06 06:17:25 INFO TaskSchedulerImpl: Removed TaskSet 311.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:25 INFO DAGScheduler: ShuffleMapStage 311
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
360.102 s
25/02/06 06:17:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:25 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 314,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:25 INFO DAGScheduler: failed: Set()
25/02/06 06:17:25 INFO TaskSetManager: Finished task 3.0 in stage 314.0 (TID 1340)
in 278 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:17:25 INFO TaskSetManager: Starting task 2.0 in stage 329.0 (TID 1368)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Finished task 4.0 in stage 317.0 (TID 1347)
in 275 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 111 to 10.183.233.191:55538
25/02/06 06:17:25 INFO TaskSetManager: Starting task 3.0 in stage 329.0 (TID 1369)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Starting task 4.0 in stage 329.0 (TID 1370)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:25 INFO TaskSetManager: Finished task 0.0 in stage 317.0 (TID 1343)
in 282 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:17:25 INFO TaskSetManager: Finished task 2.0 in stage 314.0 (TID 1339)
in 13193 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:25 INFO TaskSchedulerImpl: Removed TaskSet 314.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 111 to 10.183.232.45:60256
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 100 to 10.183.229.123:55848
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 97 to 10.183.229.123:55848
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 100 to 10.183.249.197:36834
25/02/06 06:17:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 111 to 10.183.249.197:36834
25/02/06 06:17:25 INFO DAGScheduler: ShuffleMapStage 314
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
360.067 s
25/02/06 06:17:25 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:25 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 323, ShuffleMapStage 338,
ShuffleMapStage 317, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428,
ShuffleMapStage 320, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:17:25 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:25 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 326.0 (TID 1362)
in 12044 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 5.0 in stage 329.0 (TID 1371)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 1.0 in stage 326.0 (TID 1361)
in 12047 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:17:37 INFO AsyncEventQueue: Process of event
SparkListenerJobEnd(197,1738822645306,JobSucceeded) by listener
DBCEventLoggingListener took 11.975527463s.
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 326.0 (TID 1364)
in 12047 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 326.0 (TID 1363)
in 12047 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 332.0 (TID 1372)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 332.0 (TID 1373)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 332.0 (TID 1374)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 100 to 10.183.232.45:60256
25/02/06 06:17:37 INFO BlockManagerInfo: Removed broadcast_91_piece0 on
10.183.246.72:43231 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:37 INFO BlockManagerInfo: Removed broadcast_91_piece0 on
10.183.229.123:38441 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 109 to 10.183.232.45:60256
25/02/06 06:17:37 INFO BlockManagerInfo: Removed broadcast_91_piece0 on
10.183.233.191:42923 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:37 INFO BlockManagerInfo: Removed broadcast_91_piece0 on
10.183.232.45:45579 in memory (size: 38.7 KiB, free: 18.4 GiB)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 332.0 (TID 1375)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 332.0 (TID 1376)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 335.0 (TID 1377)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 323.0 (TID 1359)
in 12130 ms on 10.183.233.191 (executor 5) (1/5)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 323.0 (TID 1358)
in 12131 ms on 10.183.233.191 (executor 5) (2/5)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 0.0 in stage 323.0 (TID 1355)
in 12229 ms on 10.183.233.191 (executor 5) (3/5)
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 109 to 10.183.233.191:55538
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 110 to 10.183.233.191:55538
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 329.0 (TID 1368)
in 12164 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 335.0 (TID 1378)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 335.0 (TID 1379)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 320.0 (TID 1351)
in 12413 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 110 to 10.183.246.72:39896
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 110 to 10.183.229.123:55848
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 335.0 (TID 1380)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 323.0 (TID 1357)
in 12189 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 335.0 (TID 1381)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 329.0 (TID 1370)
in 12179 ms on 10.183.229.123 (executor 0) (2/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 320.0 (TID 1353)
in 12431 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 5.0 in stage 335.0 (TID 1382)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 338.0 (TID 1383)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 1.0 in stage 323.0 (TID 1356)
in 12206 ms on 10.183.229.123 (executor 0) (5/5)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 323.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 323
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
369.574 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 317,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332,
ShuffleMapStage 347, ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 320,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 338.0 (TID 1384)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 0.0 in stage 326.0 (TID 1360)
in 12208 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 113 to 10.183.229.123:55848
25/02/06 06:17:37 INFO BlockManagerInfo: Removed broadcast_91_piece0 on
10.183.232.44:37705 in memory (size: 38.7 KiB, free: 20.2 GiB)
25/02/06 06:17:37 INFO CommChannelWebSocket: [session: 734823169]
onWebSocketConnect with headers: {Accept-Encoding=[gzip], Cache-Control=[no-cache],
Connection=[Upgrade], Db-Outgoing-Buffer-Throttler-Burst=[60000000], Db-Outgoing-
Buffer-Throttler-Steady-Rate=[6000000], Db-Outgoing-Buffer-Throttler-Warning-
Interval-Sec=[60], Dnc-Connection-Safe-
Flags=[enableLSPGoToDefinitionOnModule_v2=true;enableWSFSPythonModulePeek=false;jed
iAutoImportModulesList=asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,
urllib3;], Host=[10.183.232.44:6062], Pragma=[no-cache], Proxied=[true], Proxied-
Host=[10.183.232.44:6062], Sec-WebSocket-Key=[gonviBUFS7I7gFS2+DyPTg==], Sec-
WebSocket-Version=[13], Upgrade=[websocket], User-Agent=[Jetty/9.4.51.v20230217],
X-Forwarded-For=[10.2.113.226], X-Forwarded-Proto=[https]}
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 338.0 (TID 1385)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 317.0 (TID 1346)
in 12499 ms on 10.183.246.72 (executor 6) (6/6)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 317.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 113 to 10.183.246.72:39896
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 320.0 (TID 1352)
in 12489 ms on 10.183.246.72 (executor 6) (6/6)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 320.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 338.0 (TID 1386)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 113 to 10.183.233.191:55538
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 338.0 (TID 1387)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 5.0 in stage 338.0 (TID 1388)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 1.0 in stage 332.0 (TID 1373)
in 240 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 341.0 (TID 1389)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 332.0 (TID 1374)
in 240 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 341.0 (TID 1390)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 0.0 in stage 332.0 (TID 1372)
in 242 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 341.0 (TID 1391)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 341.0 (TID 1392)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 341.0 (TID 1393)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 344.0 (TID 1394)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 0.0 in stage 329.0 (TID 1366)
in 12290 ms on 10.183.249.197 (executor 7) (3/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 5.0 in stage 326.0 (TID 1365)
in 12290 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 326.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO TaskSetManager: Finished task 1.0 in stage 329.0 (TID 1367)
in 12290 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 5.0 in stage 329.0 (TID 1371)
in 248 ms on 10.183.232.45 (executor 2) (5/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 329.0 (TID 1369)
in 12280 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 329.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 113 to 10.183.232.45:60256
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 112 to 10.183.232.45:60256
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 112 to 10.183.249.197:36834
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 115 to 10.183.249.197:36834
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 344.0 (TID 1395)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 344.0 (TID 1396)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 0.0 in stage 335.0 (TID 1377)
in 190 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 335.0 (TID 1381)
in 130 ms on 10.183.229.123 (executor 0) (2/6)
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 115 to 10.183.229.123:55848
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 115 to 10.183.233.191:55538
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 317
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
370.778 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 320, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 332.0 (TID 1375)
in 212 ms on 10.183.233.191 (executor 5) (4/5)
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 320
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
370.781 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 326, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 344.0 (TID 1397)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 326
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
369.707 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 329, ShuffleMapStage 431,
ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383,
ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350,
ShuffleMapStage 365, ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362,
ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Finished task 1.0 in stage 335.0 (TID 1378)
in 174 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 344.0 (TID 1398)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 335.0 (TID 1380)
in 193 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 5.0 in stage 344.0 (TID 1399)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 332.0 (TID 1376)
in 254 ms on 10.183.233.191 (executor 5) (5/5)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 332.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO TaskSetManager: Starting task 0.0 in stage 347.0 (TID 1400)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 329
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
369.323 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 332, ShuffleMapStage 347, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 332
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
369.347 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 335, ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410,
ShuffleMapStage 425, ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398,
ShuffleMapStage 377, ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365,
ShuffleMapStage 344, ShuffleMapStage 338, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 353, ShuffleMapStage 347, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 338.0 (TID 1385)
in 194 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 1.0 in stage 347.0 (TID 1401)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:17:37 INFO TaskSetManager: Finished task 3.0 in stage 338.0 (TID 1386)
in 181 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 2.0 in stage 347.0 (TID 1402)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:37 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 114 to 10.183.246.72:39896
25/02/06 06:17:37 INFO TaskSetManager: Finished task 5.0 in stage 335.0 (TID 1382)
in 254 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 3.0 in stage 347.0 (TID 1403)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:37 INFO CommChannelWebSocket: [session: 3030842] onWebSocketConnect
with headers: {Accept-Encoding=[gzip], Cache-Control=[no-cache],
Connection=[Upgrade], Db-Outgoing-Buffer-Throttler-Burst=[60000000], Db-Outgoing-
Buffer-Throttler-Steady-Rate=[6000000], Db-Outgoing-Buffer-Throttler-Warning-
Interval-Sec=[60], Dnc-Connection-Safe-
Flags=[enableLSPGoToDefinitionOnModule_v2=true;enableWSFSPythonModulePeek=false;jed
iAutoImportModulesList=asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,
urllib3;], Host=[10.183.232.44:6062], Pragma=[no-cache], Proxied=[true], Proxied-
Host=[10.183.232.44:6062], Sec-WebSocket-Key=[o3d7fB+dlnoZWXOupQ3lUg==], Sec-
WebSocket-Version=[13], Upgrade=[websocket], User-Agent=[Jetty/9.4.51.v20230217],
X-Forwarded-For=[10.2.113.226], X-Forwarded-Proto=[https]}
25/02/06 06:17:37 INFO TaskSetManager: Finished task 2.0 in stage 335.0 (TID 1379)
in 276 ms on 10.183.246.72 (executor 6) (6/6)
25/02/06 06:17:37 INFO TaskSchedulerImpl: Removed TaskSet 335.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:37 INFO TaskSetManager: Starting task 4.0 in stage 347.0 (TID 1404)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:17:37 INFO DAGScheduler: ShuffleMapStage 335
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
369.379 s
25/02/06 06:17:37 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:37 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 338, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 347, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362, ShuffleMapStage 341)
25/02/06 06:17:37 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:37 INFO DAGScheduler: failed: Set()
25/02/06 06:17:37 INFO TaskSetManager: Finished task 4.0 in stage 344.0 (TID 1398)
in 120 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:17:37 INFO TaskSetManager: Starting task 5.0 in stage 347.0 (TID 1405)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 5.0 in stage 338.0 (TID 1388)
in 11027 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 341.0 (TID 1390)
in 11026 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 4.0 in stage 338.0 (TID 1387)
in 11028 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 341.0 (TID 1389)
in 11027 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 341.0 (TID 1392)
in 11025 ms on 10.183.249.197 (executor 7) (3/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 4.0 in stage 341.0 (TID 1393)
in 11026 ms on 10.183.249.197 (executor 7) (4/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 344.0 (TID 1394)
in 11026 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 341.0 (TID 1391)
in 11026 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 341.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 350.0 (TID 1406)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 350.0 (TID 1407)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 350.0 (TID 1408)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 350.0 (TID 1409)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 350.0 (TID 1410)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 353.0 (TID 1411)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 353.0 (TID 1412)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 353.0 (TID 1413)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 116 to 10.183.232.45:60256
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 116 to 10.183.249.197:36834
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 120 to 10.183.249.197:36834
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 341
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
379.064 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 338, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353,
ShuffleMapStage 347, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 344.0 (TID 1396)
in 11039 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 353.0 (TID 1414)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 455775985]
Stop MessageSendTask
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 114 to 10.183.233.191:55538
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 353.0 (TID 1415)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 5.0 in stage 353.0 (TID 1416)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 338.0 (TID 1384)
in 11152 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 344.0 (TID 1397)
in 11018 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:17:48 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:48 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:48 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:48 INFO ProgressReporter$: Reporting partial results for running
commands: 5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9
[20 occurrences]
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 114 to 10.183.229.123:55848
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 356.0 (TID 1417)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 356.0 (TID 1418)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 338.0 (TID 1383)
in 11226 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 338.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO TaskSetManager: Finished task 5.0 in stage 344.0 (TID 1399)
in 11051 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 344.0 (TID 1395)
in 11115 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 344.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 356.0 (TID 1419)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 117 to 10.183.229.123:55848
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 3030842]
Start MessageSendTask
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 3030842] Stop
MessageSendTask
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 734823169]
Start MessageSendTask
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 338
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
379.230 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 120 to 10.183.233.191:55538
25/02/06 06:17:48 INFO CommChannelWebSocket: [session: 326746184]
onWebSocketConnect with headers: {Accept-Encoding=[gzip], Cache-Control=[no-cache],
Connection=[Upgrade], Db-Outgoing-Buffer-Throttler-Burst=[60000000], Db-Outgoing-
Buffer-Throttler-Steady-Rate=[6000000], Db-Outgoing-Buffer-Throttler-Warning-
Interval-Sec=[60], Dnc-Connection-Safe-
Flags=[enableLSPGoToDefinitionOnModule_v2=true;enableWSFSPythonModulePeek=false;jed
iAutoImportModulesList=asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,
urllib3;], Host=[10.183.232.44:6062], Pragma=[no-cache], Proxied=[true], Proxied-
Host=[10.183.232.44:6062], Sec-WebSocket-Key=[4DLbr804LZt9FRxCnCeqZA==], Sec-
WebSocket-Version=[13], Upgrade=[websocket], User-Agent=[Jetty/9.4.51.v20230217],
X-Forwarded-For=[10.2.113.226], X-Forwarded-Proto=[https]}
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 344,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 347,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 734823169]
Stop MessageSendTask
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 326746184]
Start MessageSendTask
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 347.0 (TID 1402)
in 11093 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 4.0 in stage 347.0 (TID 1404)
in 11074 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 356.0 (TID 1420)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 356.0 (TID 1421)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO CommChannelWebSocket: [session: 455775985] onWebSocketClose
with statusCode: 1001, reason: java.util.concurrent.TimeoutException: Idle timeout
expired: 300000/300000 ms
25/02/06 06:17:48 INFO CommChannelWebSocket: [session: 734823169] onWebSocketClose
with statusCode: 1001, reason: Web socket destroyed before connection established
25/02/06 06:17:48 INFO CommChannelWebSocket: [session: 3030842] onWebSocketClose
with statusCode: 1006, reason: Disconnected
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 344
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
378.033 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 353, ShuffleMapStage 347, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 117 to 10.183.246.72:39896
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 347.0 (TID 1401)
in 11185 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 347.0 (TID 1403)
in 11162 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 4.0 in stage 350.0 (TID 1410)
in 292 ms on 10.183.249.197 (executor 7) (1/5)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 359.0 (TID 1422)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 359.0 (TID 1423)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 359.0 (TID 1424)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO OutgoingDirectNotebookMessageBuffer: [session: 326746184]
Stop MessageSendTask
25/02/06 06:17:48 INFO CommChannelWebSocket: [session: 326746184] onWebSocketClose
with statusCode: 1001, reason: Web socket destroyed before connection established
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 119 to 10.183.246.72:39896
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 359.0 (TID 1425)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 4.0 in stage 353.0 (TID 1415)
in 298 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 359.0 (TID 1426)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 353.0 (TID 1414)
in 304 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 362.0 (TID 1427)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 5.0 in stage 353.0 (TID 1416)
in 307 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 119 to 10.183.233.191:55538
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 362.0 (TID 1428)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 347.0 (TID 1400)
in 11294 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 118 to 10.183.233.191:55538
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 119 to 10.183.249.197:36834
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 362.0 (TID 1429)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 353.0 (TID 1413)
in 367 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 353.0 (TID 1412)
in 367 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 362.0 (TID 1430)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 362.0 (TID 1431)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 353.0 (TID 1411)
in 368 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 353.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 353
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
377.022 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 350, ShuffleMapStage 365, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359,
ShuffleMapStage 374, ShuffleMapStage 347, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 350.0 (TID 1407)
in 369 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 350.0 (TID 1408)
in 370 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 350.0 (TID 1409)
in 374 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 5.0 in stage 362.0 (TID 1432)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 365.0 (TID 1433)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 1.0 in stage 365.0 (TID 1434)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Starting task 2.0 in stage 365.0 (TID 1435)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 118 to 10.183.249.197:36834
25/02/06 06:17:48 INFO TaskSetManager: Finished task 0.0 in stage 350.0 (TID 1406)
in 385 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 350.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 350
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
378.118 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 365, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 347, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368,
ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO TaskSetManager: Finished task 2.0 in stage 356.0 (TID 1419)
in 269 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:17:48 INFO TaskSetManager: Finished task 5.0 in stage 347.0 (TID 1405)
in 11243 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:17:48 INFO TaskSchedulerImpl: Removed TaskSet 347.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:48 INFO TaskSetManager: Starting task 3.0 in stage 365.0 (TID 1436)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:48 INFO DAGScheduler: ShuffleMapStage 347
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
378.143 s
25/02/06 06:17:48 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:48 INFO TaskSetManager: Starting task 4.0 in stage 365.0 (TID 1437)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:48 INFO DAGScheduler: running: Set(ShuffleMapStage 356,
ShuffleMapStage 437, ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425,
ShuffleMapStage 404, ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377,
ShuffleMapStage 371, ShuffleMapStage 365, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:48 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:48 INFO DAGScheduler: failed: Set()
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 122 to 10.183.232.45:60256
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 118 to 10.183.232.45:60256
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 122 to 10.183.229.123:55848
25/02/06 06:17:48 INFO TaskSetManager: Finished task 3.0 in stage 356.0 (TID 1420)
in 212 ms on 10.183.246.72 (executor 6) (2/5)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 5.0 in stage 365.0 (TID 1438)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:48 INFO TaskSetManager: Finished task 1.0 in stage 356.0 (TID 1418)
in 308 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:17:48 INFO TaskSetManager: Starting task 0.0 in stage 368.0 (TID 1439)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 122 to 10.183.246.72:39896
25/02/06 06:17:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 121 to 10.183.229.123:55848
25/02/06 06:17:49 INFO TaskSetManager: Starting task 1.0 in stage 368.0 (TID 1440)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:49 INFO TaskSetManager: Finished task 0.0 in stage 356.0 (TID 1417)
in 330 ms on 10.183.229.123 (executor 0) (4/5)
25/02/06 06:17:49 INFO TaskSetManager: Starting task 2.0 in stage 368.0 (TID 1441)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:17:49 INFO TaskSetManager: Finished task 4.0 in stage 356.0 (TID 1421)
in 245 ms on 10.183.246.72 (executor 6) (5/5)
25/02/06 06:17:49 INFO TaskSchedulerImpl: Removed TaskSet 356.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:49 INFO DAGScheduler: ShuffleMapStage 356
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
377.103 s
25/02/06 06:17:49 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:49 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 365, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 359, ShuffleMapStage 374, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389, ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:49 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:49 INFO DAGScheduler: failed: Set()
25/02/06 06:17:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 121 to 10.183.246.72:39896
25/02/06 06:17:49 INFO TaskSetManager: Finished task 1.0 in stage 359.0 (TID 1423)
in 227 ms on 10.183.246.72 (executor 6) (1/5)
25/02/06 06:17:49 INFO TaskSetManager: Starting task 3.0 in stage 368.0 (TID 1442)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 4.0 in stage 359.0 (TID 1426)
in 10751 ms on 10.183.233.191 (executor 5) (2/5)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 368.0 (TID 1443)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 371.0 (TID 1444)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 359.0 (TID 1425)
in 10756 ms on 10.183.233.191 (executor 5) (3/5)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 1.0 in stage 371.0 (TID 1445)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 2.0 in stage 371.0 (TID 1446)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 0.0 in stage 362.0 (TID 1427)
in 10748 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 362.0 (TID 1428)
in 10738 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:17:59 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:59 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:59 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:59 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:17:59 ERROR RetryingHMSHandler: Retrying HMSHandler after 2000 ms
(attempt 2 of 10) with error: javax.jdo.JDODataStoreException: HikariPool-1 -
Connection is not available, request timed out after 34847ms.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(Nucleus
JDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
NestedThrowablesStackTrace:
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not
available, request timed out after 34847ms.
at
com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
at
org.datanucleus.store.rdbms.ConnectionProviderPriorityList.getConnection(Connection
ProviderPriorityList.java:57)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnecti
on(ConnectionFactoryImpl.java:402)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getXAResour
ce(ConnectionFactoryImpl.java:361)
at
org.datanucleus.store.connection.ConnectionManagerImpl.allocateConnection(Connectio
nManagerImpl.java:316)
at
org.datanucleus.store.connection.AbstractConnectionFactory.getConnection(AbstractCo
nnectionFactory.java:84)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
347)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
310)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:591)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1855)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 121 to 10.183.233.191:55538
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 123 to 10.183.233.191:55538
25/02/06 06:17:59 INFO TaskSetManager: Finished task 4.0 in stage 365.0 (TID 1437)
in 10783 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 5.0 in stage 362.0 (TID 1432)
in 10802 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 365.0 (TID 1435)
in 10794 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 368.0 (TID 1440)
in 10732 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 365.0 (TID 1434)
in 10795 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 0.0 in stage 359.0 (TID 1422)
in 10884 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 368.0 (TID 1442)
in 10656 ms on 10.183.246.72 (executor 6) (2/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 0.0 in stage 368.0 (TID 1439)
in 10755 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 368.0 (TID 1441)
in 10727 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 359.0 (TID 1424)
in 10884 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:17:59 INFO TaskSchedulerImpl: Removed TaskSet 359.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:59 INFO TaskSetManager: Finished task 4.0 in stage 362.0 (TID 1431)
in 10810 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 365.0 (TID 1436)
in 10787 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 362.0 (TID 1430)
in 10812 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:17:59 INFO DAGScheduler: ShuffleMapStage 359
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
387.816 s
25/02/06 06:17:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:59 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 365, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 374, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368, ShuffleMapStage 362)
25/02/06 06:17:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:59 INFO DAGScheduler: failed: Set()
25/02/06 06:17:59 INFO TaskSetManager: Starting task 3.0 in stage 371.0 (TID 1447)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 371.0 (TID 1448)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 5.0 in stage 371.0 (TID 1449)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 374.0 (TID 1450)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 1.0 in stage 374.0 (TID 1451)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 2.0 in stage 374.0 (TID 1452)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 3.0 in stage 374.0 (TID 1453)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 374.0 (TID 1454)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 377.0 (TID 1455)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 1.0 in stage 377.0 (TID 1456)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 2.0 in stage 377.0 (TID 1457)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 3.0 in stage 377.0 (TID 1458)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 377.0 (TID 1459)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 5.0 in stage 377.0 (TID 1460)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 380.0 (TID 1461)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 1.0 in stage 380.0 (TID 1462)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 123 to 10.183.229.123:55848
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 125 to 10.183.229.123:55848
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 124 to 10.183.229.123:55848
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 125 to 10.183.246.72:39896
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 126 to 10.183.246.72:39896
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 124 to 10.183.246.72:39896
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 125 to 10.183.232.45:60256
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 123 to 10.183.232.45:60256
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 124 to 10.183.232.45:60256
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 123 to 10.183.249.197:36834
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 125 to 10.183.249.197:36834
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 124 to 10.183.249.197:36834
25/02/06 06:17:59 INFO TaskSetManager: Finished task 5.0 in stage 365.0 (TID 1438)
in 10800 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 362.0 (TID 1429)
in 10854 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:17:59 INFO TaskSchedulerImpl: Removed TaskSet 362.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:59 INFO TaskSetManager: Finished task 0.0 in stage 365.0 (TID 1433)
in 10838 ms on 10.183.232.45 (executor 2) (6/6)
25/02/06 06:17:59 INFO TaskSchedulerImpl: Removed TaskSet 365.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:59 INFO DAGScheduler: ShuffleMapStage 362
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
387.843 s
25/02/06 06:17:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:59 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 365, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 380, ShuffleMapStage 374, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389,
ShuffleMapStage 368)
25/02/06 06:17:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:59 INFO DAGScheduler: failed: Set()
25/02/06 06:17:59 INFO DAGScheduler: ShuffleMapStage 365
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
385.690 s
25/02/06 06:17:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:59 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 374, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389, ShuffleMapStage 368)
25/02/06 06:17:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:59 INFO DAGScheduler: failed: Set()
25/02/06 06:17:59 INFO TaskSetManager: Finished task 4.0 in stage 368.0 (TID 1443)
in 280 ms on 10.183.233.191 (executor 5) (5/5)
25/02/06 06:17:59 INFO TaskSchedulerImpl: Removed TaskSet 368.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:17:59 INFO DAGScheduler: ShuffleMapStage 368
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
385.807 s
25/02/06 06:17:59 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:17:59 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 371,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380,
ShuffleMapStage 374, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389)
25/02/06 06:17:59 INFO DAGScheduler: waiting: Set()
25/02/06 06:17:59 INFO DAGScheduler: failed: Set()
25/02/06 06:17:59 INFO TaskSetManager: Starting task 2.0 in stage 380.0 (TID 1463)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 5.0 in stage 371.0 (TID 1449)
in 204 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 3.0 in stage 380.0 (TID 1464)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 5.0 in stage 377.0 (TID 1460)
in 205 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 380.0 (TID 1465)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 126 to 10.183.233.191:55538
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 126 to 10.183.229.123:55848
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 380.0 (TID 1462)
in 215 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 5.0 in stage 380.0 (TID 1466)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 371.0 (TID 1447)
in 222 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 383.0 (TID 1467)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 127 to 10.183.232.45:60256
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 371.0 (TID 1445)
in 316 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 1.0 in stage 383.0 (TID 1468)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 2.0 in stage 383.0 (TID 1469)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 0.0 in stage 380.0 (TID 1461)
in 235 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 377.0 (TID 1457)
in 236 ms on 10.183.229.123 (executor 0) (2/6)
25/02/06 06:17:59 INFO TaskSetManager: Starting task 3.0 in stage 383.0 (TID 1470)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Starting task 4.0 in stage 383.0 (TID 1471)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 2.0 in stage 374.0 (TID 1452)
in 238 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:17:59 INFO TaskSetManager: Finished task 1.0 in stage 374.0 (TID 1451)
in 241 ms on 10.183.249.197 (executor 7) (2/5)
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 127 to 10.183.246.72:39896
25/02/06 06:17:59 INFO TaskSetManager: Starting task 5.0 in stage 383.0 (TID 1472)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 127 to 10.183.229.123:55848
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 127 to 10.183.233.191:55538
25/02/06 06:17:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 127 to 10.183.249.197:36834
25/02/06 06:17:59 INFO TaskSetManager: Starting task 0.0 in stage 386.0 (TID 1473)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:17:59 INFO TaskSetManager: Finished task 3.0 in stage 377.0 (TID 1458)
in 260 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 2.0 in stage 371.0 (TID 1446)
in 346 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 3.0 in stage 374.0 (TID 1453)
in 265 ms on 10.183.246.72 (executor 6) (3/5)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 386.0 (TID 1474)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 2.0 in stage 386.0 (TID 1475)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 133 to 10.183.246.72:39896
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 374.0 (TID 1454)
in 268 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 3.0 in stage 386.0 (TID 1476)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 4.0 in stage 386.0 (TID 1477)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 377.0 (TID 1455)
in 272 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 133 to 10.183.233.191:55538
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 133 to 10.183.232.45:60256
25/02/06 06:18:00 INFO TaskSetManager: Starting task 5.0 in stage 386.0 (TID 1478)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 371.0 (TID 1444)
in 361 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 0.0 in stage 389.0 (TID 1479)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 371.0 (TID 1448)
in 290 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 371.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 371
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
385.889 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 374,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 131 to 10.183.249.197:36834
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 389.0 (TID 1480)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 374.0 (TID 1450)
in 301 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 374.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 374
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
384.026 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 377, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392,
ShuffleMapStage 407, ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 377.0 (TID 1459)
in 304 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 2.0 in stage 389.0 (TID 1481)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 131 to 10.183.232.45:60256
25/02/06 06:18:00 INFO TaskSetManager: Finished task 1.0 in stage 377.0 (TID 1456)
in 338 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 377.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO TaskSetManager: Starting task 3.0 in stage 389.0 (TID 1482)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 377
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
384.025 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 380, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO TaskSetManager: Starting task 4.0 in stage 389.0 (TID 1483)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 5.0 in stage 389.0 (TID 1484)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 5.0 in stage 380.0 (TID 1466)
in 211 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 1.0 in stage 383.0 (TID 1468)
in 194 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 2.0 in stage 380.0 (TID 1463)
in 233 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 5.0 in stage 383.0 (TID 1472)
in 189 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 2.0 in stage 383.0 (TID 1469)
in 196 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 380.0 (TID 1465)
in 226 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 383.0 (TID 1471)
in 194 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 3.0 in stage 380.0 (TID 1464)
in 229 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 380.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO TaskSetManager: Finished task 3.0 in stage 383.0 (TID 1470)
in 196 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 131 to 10.183.246.72:39896
25/02/06 06:18:00 INFO TaskSetManager: Finished task 2.0 in stage 386.0 (TID 1475)
in 171 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 0.0 in stage 392.0 (TID 1485)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 392.0 (TID 1486)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 2.0 in stage 392.0 (TID 1487)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 3.0 in stage 392.0 (TID 1488)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 380
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
384.110 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 383, ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407,
ShuffleMapStage 386, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401,
ShuffleMapStage 416, ShuffleMapStage 395, ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO TaskSetManager: Starting task 4.0 in stage 392.0 (TID 1489)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 0.0 in stage 395.0 (TID 1490)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 395.0 (TID 1491)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 2.0 in stage 395.0 (TID 1492)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 386.0 (TID 1473)
in 184 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 132 to 10.183.246.72:39896
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 131 to 10.183.233.191:55538
25/02/06 06:18:00 INFO TaskSetManager: Starting task 3.0 in stage 395.0 (TID 1493)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 132 to 10.183.249.197:36834
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 132 to 10.183.233.191:55538
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 132 to 10.183.229.123:55848
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 130 to 10.183.229.123:55848
25/02/06 06:18:00 INFO TaskSetManager: Finished task 3.0 in stage 389.0 (TID 1482)
in 135 ms on 10.183.249.197 (executor 7) (1/6)
25/02/06 06:18:00 INFO TaskSetManager: Starting task 4.0 in stage 395.0 (TID 1494)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 5.0 in stage 395.0 (TID 1495)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 0.0 in stage 398.0 (TID 1496)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 398.0 (TID 1497)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 2.0 in stage 398.0 (TID 1498)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 3.0 in stage 398.0 (TID 1499)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 4.0 in stage 398.0 (TID 1500)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 0.0 in stage 401.0 (TID 1501)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Starting task 1.0 in stage 401.0 (TID 1502)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:18:00 INFO TaskSetManager: Finished task 3.0 in stage 386.0 (TID 1476)
in 301 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 383.0 (TID 1467)
in 349 ms on 10.183.232.45 (executor 2) (6/6)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 383.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 130 to 10.183.246.72:39896
25/02/06 06:18:00 INFO TaskSetManager: Finished task 0.0 in stage 389.0 (TID 1479)
in 283 ms on 10.183.249.197 (executor 7) (2/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 5.0 in stage 386.0 (TID 1478)
in 295 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 1.0 in stage 386.0 (TID 1474)
in 307 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 130 to 10.183.249.197:36834
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 383
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
384.201 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 386,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416,
ShuffleMapStage 395, ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 129 to 10.183.233.191:55538
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 128 to 10.183.232.45:60256
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 130 to 10.183.232.45:60256
25/02/06 06:18:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 128 to 10.183.249.197:36834
25/02/06 06:18:00 INFO TaskSetManager: Finished task 4.0 in stage 386.0 (TID 1477)
in 308 ms on 10.183.232.45 (executor 2) (6/6)
25/02/06 06:18:00 INFO TaskSchedulerImpl: Removed TaskSet 386.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:00 INFO TaskSetManager: Finished task 1.0 in stage 389.0 (TID 1480)
in 281 ms on 10.183.232.45 (executor 2) (3/6)
25/02/06 06:18:00 INFO TaskSetManager: Finished task 2.0 in stage 389.0 (TID 1481)
in 276 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:18:00 INFO DAGScheduler: ShuffleMapStage 386
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
383.157 s
25/02/06 06:18:00 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:00 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395,
ShuffleMapStage 389)
25/02/06 06:18:00 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:00 INFO DAGScheduler: failed: Set()
25/02/06 06:18:00 INFO ShufflePartitionsUtil: For shuffle(156), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:18:11 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:18:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:11 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 392.0 (TID 1488)
in 11163 ms on 10.183.249.197 (executor 7) (1/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 401.0 (TID 1503)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 389.0 (TID 1483)
in 11188 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 401.0 (TID 1504)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 129 to 10.183.249.197:36834
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 129 to 10.183.246.72:39896
25/02/06 06:18:11 WARN DBRDebuggerEventReporter: Driver/10.183.232.44 paused the
JVM process 70 seconds during the past 120 seconds (58.59%) because of GC. We
observed 4 such issue(s) since 2025-02-06T06:13:03.371Z.
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(161), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 401.0 (TID 1502)
in 11101 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 401.0 (TID 1505)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 392.0 (TID 1485)
in 11238 ms on 10.183.233.191 (executor 5) (2/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 401.0 (TID 1506)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(147), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 395.0 (TID 1495)
in 11126 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 392.0 (TID 1489)
in 11252 ms on 10.183.229.123 (executor 0) (3/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 404.0 (TID 1507)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 404.0 (TID 1508)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 395.0 (TID 1493)
in 11248 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 404.0 (TID 1509)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 134 to 10.183.232.45:60256
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 134 to 10.183.229.123:55848
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 134 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 398.0 (TID 1496)
in 11149 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 404.0 (TID 1510)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 404.0 (TID 1511)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 389.0 (TID 1484)
in 11303 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 389.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 389
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
394.279 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416, ShuffleMapStage 395)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 134 to 10.183.233.191:55538
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 404.0 (TID 1512)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 392.0 (TID 1487)
in 11310 ms on 10.183.246.72 (executor 6) (4/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 407.0 (TID 1513)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 395.0 (TID 1494)
in 11283 ms on 10.183.249.197 (executor 7) (3/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 407.0 (TID 1514)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 395.0 (TID 1492)
in 11317 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 407.0 (TID 1515)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 401.0 (TID 1505)
in 90 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 398.0 (TID 1498)
in 11197 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(168), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 407.0 (TID 1516)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 135 to 10.183.229.123:55848
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 407.0 (TID 1517)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 398.0 (TID 1497)
in 11200 ms on 10.183.232.45 (executor 2) (3/5)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 135 to 10.183.249.197:36834
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 135 to 10.183.232.45:60256
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 135 to 10.183.233.191:55538
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 407.0 (TID 1518)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 395.0 (TID 1490)
in 11341 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 395.0 (TID 1491)
in 11344 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 395.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 410.0 (TID 1519)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 395
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
394.257 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 392, ShuffleMapStage 407, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 401, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 392.0 (TID 1486)
in 11349 ms on 10.183.246.72 (executor 6) (5/5)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 392.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 410.0 (TID 1520)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 392
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
394.313 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 407, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 401, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 136 to 10.183.229.123:55848
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 136 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 398.0 (TID 1500)
in 11230 ms on 10.183.249.197 (executor 7) (4/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 410.0 (TID 1521)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 136 to 10.183.249.197:36834
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 401.0 (TID 1501)
in 11242 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 401.0 (TID 1506)
in 137 ms on 10.183.233.191 (executor 5) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 401.0 (TID 1504)
in 197 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 410.0 (TID 1522)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 410.0 (TID 1523)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 413.0 (TID 1524)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 136 to 10.183.233.191:55538
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 137 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 413.0 (TID 1525)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 401.0 (TID 1503)
in 229 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 401.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 401
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
394.257 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 398, ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434,
ShuffleMapStage 413, ShuffleMapStage 407, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 413.0 (TID 1526)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 404.0 (TID 1507)
in 142 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 413.0 (TID 1527)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 137 to 10.183.249.197:36834
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 137 to 10.183.232.45:60256
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 404.0 (TID 1510)
in 133 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 413.0 (TID 1528)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 398.0 (TID 1499)
in 11294 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 398.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 398
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
394.294 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 407, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 413.0 (TID 1529)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 404.0 (TID 1509)
in 171 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(155), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 416.0 (TID 1530)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 138 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 404.0 (TID 1512)
in 157 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 416.0 (TID 1531)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 407.0 (TID 1514)
in 150 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 416.0 (TID 1532)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 404.0 (TID 1508)
in 217 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 138 to 10.183.229.123:55848
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 416.0 (TID 1533)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 407.0 (TID 1516)
in 159 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 416.0 (TID 1534)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 407.0 (TID 1518)
in 154 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 138 to 10.183.233.191:55538
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 407.0 (TID 1515)
in 209 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 416.0 (TID 1535)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 138 to 10.183.232.45:60256
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 407.0 (TID 1513)
in 234 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 419.0 (TID 1536)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 413.0 (TID 1527)
in 152 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 407.0 (TID 1517)
in 227 ms on 10.183.232.45 (executor 2) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 407.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 413.0 (TID 1526)
in 159 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 407
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
393.067 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 404,
ShuffleMapStage 440, ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413,
ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 419.0 (TID 1537)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 419.0 (TID 1538)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 419.0 (TID 1539)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 410.0 (TID 1521)
in 203 ms on 10.183.249.197 (executor 7) (1/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 419.0 (TID 1540)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 139 to 10.183.232.45:60256
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 139 to 10.183.249.197:36834
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 410.0 (TID 1519)
in 225 ms on 10.183.229.123 (executor 0) (2/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 422.0 (TID 1541)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 410.0 (TID 1520)
in 223 ms on 10.183.246.72 (executor 6) (3/5)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 413.0 (TID 1529)
in 144 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 413.0 (TID 1524)
in 197 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 422.0 (TID 1542)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 422.0 (TID 1543)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 422.0 (TID 1544)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 404.0 (TID 1511)
in 282 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 404.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 410.0 (TID 1523)
in 200 ms on 10.183.233.191 (executor 5) (4/5)
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 410.0 (TID 1522)
in 201 ms on 10.183.233.191 (executor 5) (5/5)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 410.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 422.0 (TID 1545)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 422.0 (TID 1546)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 425.0 (TID 1547)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 140 to 10.183.229.123:55848
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 404
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
393.158 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 410, ShuffleMapStage 425, ShuffleMapStage 440,
ShuffleMapStage 419, ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 428,
ShuffleMapStage 422, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 410
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
393.077 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 425, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 413, ShuffleMapStage 428, ShuffleMapStage 422,
ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 142 to 10.183.233.191:55538
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 140 to 10.183.233.191:55538
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(149), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 140 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 425.0 (TID 1548)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 416.0 (TID 1530)
in 180 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 142 to 10.183.246.72:39896
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 425.0 (TID 1549)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 4.0 in stage 413.0 (TID 1528)
in 229 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 142 to 10.183.249.197:36834
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 425.0 (TID 1550)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 413.0 (TID 1525)
in 280 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:11 INFO TaskSchedulerImpl: Removed TaskSet 413.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:11 INFO DAGScheduler: ShuffleMapStage 413
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
392.095 s
25/02/06 06:18:11 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:11 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 425, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 428, ShuffleMapStage 422, ShuffleMapStage 416)
25/02/06 06:18:11 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:11 INFO DAGScheduler: failed: Set()
25/02/06 06:18:11 INFO ShufflePartitionsUtil: For shuffle(171), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:11 INFO TaskSetManager: Starting task 4.0 in stage 425.0 (TID 1551)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 2.0 in stage 419.0 (TID 1538)
in 145 ms on 10.183.232.45 (executor 2) (1/5)
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 142 to 10.183.232.45:60256
25/02/06 06:18:11 INFO TaskSetManager: Finished task 1.0 in stage 416.0 (TID 1531)
in 250 ms on 10.183.229.123 (executor 0) (2/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 5.0 in stage 425.0 (TID 1552)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 142 to 10.183.229.123:55848
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 419.0 (TID 1539)
in 174 ms on 10.183.232.45 (executor 2) (2/5)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 0.0 in stage 428.0 (TID 1553)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 141 to 10.183.232.45:60256
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 422.0 (TID 1541)
in 171 ms on 10.183.229.123 (executor 0) (1/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 1.0 in stage 428.0 (TID 1554)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 3.0 in stage 416.0 (TID 1533)
in 261 ms on 10.183.233.191 (executor 5) (3/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 2.0 in stage 428.0 (TID 1555)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:18:11 INFO TaskSetManager: Finished task 5.0 in stage 416.0 (TID 1535)
in 217 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:18:11 INFO TaskSetManager: Starting task 3.0 in stage 428.0 (TID 1556)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:18:11 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 141 to 10.183.229.123:55848
25/02/06 06:18:11 INFO TaskSetManager: Finished task 0.0 in stage 419.0 (TID 1536)
in 197 ms on 10.183.249.197 (executor 7) (3/5)
25/02/06 06:18:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 141 to 10.183.233.191:55538
25/02/06 06:18:22 INFO TaskSetManager: Starting task 4.0 in stage 428.0 (TID 1557)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 141 to 10.183.249.197:36834
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(160), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO ConcurrentBag: ConcurrentBag has been closed, ignoring add()
25/02/06 06:18:22 INFO TaskSetManager: Starting task 5.0 in stage 428.0 (TID 1558)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 0.0 in stage 431.0 (TID 1559)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 1.0 in stage 431.0 (TID 1560)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 2.0 in stage 431.0 (TID 1561)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 3.0 in stage 431.0 (TID 1562)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 4.0 in stage 431.0 (TID 1563)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 0.0 in stage 434.0 (TID 1564)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 1.0 in stage 434.0 (TID 1565)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 2.0 in stage 434.0 (TID 1566)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 3.0 in stage 434.0 (TID 1567)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 4.0 in stage 434.0 (TID 1568)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 5.0 in stage 434.0 (TID 1569)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 0.0 in stage 437.0 (TID 1570)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 1.0 in stage 437.0 (TID 1571)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 2.0 in stage 437.0 (TID 1572)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 422.0 (TID 1544)
in 10642 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 422.0 (TID 1542)
in 10642 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 422.0 (TID 1543)
in 10642 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 425.0 (TID 1548)
in 10576 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 422.0 (TID 1546)
in 10640 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 422.0 (TID 1545)
in 10640 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 422.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 425.0 (TID 1547)
in 10641 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 419.0 (TID 1537)
in 10659 ms on 10.183.232.45 (executor 2) (4/5)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 416.0 (TID 1532)
in 10750 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 419.0 (TID 1540)
in 10660 ms on 10.183.249.197 (executor 7) (5/5)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 419.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 425.0 (TID 1550)
in 10550 ms on 10.183.249.197 (executor 7) (3/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 428.0 (TID 1555)
in 10476 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:18:22 INFO TaskSetManager: Starting task 3.0 in stage 437.0 (TID 1573)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 425.0 (TID 1551)
in 10521 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:18:22 INFO TaskSetManager: Starting task 4.0 in stage 437.0 (TID 1574)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 428.0 (TID 1556)
in 10475 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 422
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
400.698 s
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 143 to 10.183.232.45:60256
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 425, ShuffleMapStage 440, ShuffleMapStage 419,
ShuffleMapStage 434, ShuffleMapStage 428, ShuffleMapStage 416)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 146 to 10.183.232.45:60256
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 419
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
400.726 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 425, ShuffleMapStage 440, ShuffleMapStage 434,
ShuffleMapStage 428, ShuffleMapStage 416)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 141 to 10.183.233.191:55538
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 143 to 10.183.233.191:55538
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 146 to 10.183.233.191:55538
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 425.0 (TID 1552)
in 10508 ms on 10.183.229.123 (executor 0) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 425.0 (TID 1549)
in 10578 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 425.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 146 to 10.183.246.72:39896
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 425
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
400.703 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 440, ShuffleMapStage 434, ShuffleMapStage 428,
ShuffleMapStage 416)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 145 to 10.183.246.72:39896
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 143 to 10.183.246.72:39896
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 416.0 (TID 1534)
in 10738 ms on 10.183.229.123 (executor 0) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 416.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 145 to 10.183.249.197:36834
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 146 to 10.183.249.197:36834
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 416
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
402.638 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 440, ShuffleMapStage 434, ShuffleMapStage 428)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 145 to 10.183.229.123:55848
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 143 to 10.183.229.123:55848
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 146 to 10.183.229.123:55848
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 145 to 10.183.232.45:60256
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 143 to 10.183.249.197:36834
25/02/06 06:18:22 INFO TaskSetManager: Starting task 5.0 in stage 437.0 (TID 1575)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 428.0 (TID 1554)
in 10496 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(164), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO TaskSetManager: Starting task 0.0 in stage 440.0 (TID 1576)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Starting task 1.0 in stage 440.0 (TID 1577)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 428.0 (TID 1553)
in 10552 ms on 10.183.232.45 (executor 2) (4/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 428.0 (TID 1557)
in 148 ms on 10.183.249.197 (executor 7) (5/6)
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 144 to 10.183.232.45:60256
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 144 to 10.183.249.197:36834
25/02/06 06:18:22 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(162), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO TaskSetManager: Starting task 2.0 in stage 440.0 (TID 1578)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 428.0 (TID 1558)
in 130 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 428.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 428
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
400.798 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 431, ShuffleMapStage 440, ShuffleMapStage 434)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 144 to 10.183.233.191:55538
25/02/06 06:18:22 INFO TaskSetManager: Starting task 3.0 in stage 440.0 (TID 1579)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 437.0 (TID 1571)
in 150 ms on 10.183.246.72 (executor 6) (1/6)
25/02/06 06:18:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 144 to 10.183.246.72:39896
25/02/06 06:18:22 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:22 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:22 INFO TaskSetManager: Starting task 4.0 in stage 440.0 (TID 1580)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 434.0 (TID 1569)
in 200 ms on 10.183.233.191 (executor 5) (1/6)
25/02/06 06:18:22 INFO TaskSetManager: Starting task 5.0 in stage 440.0 (TID 1581)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 437.0 (TID 1572)
in 205 ms on 10.183.246.72 (executor 6) (2/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 434.0 (TID 1565)
in 211 ms on 10.183.232.45 (executor 2) (2/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 437.0 (TID 1573)
in 206 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 434.0 (TID 1566)
in 225 ms on 10.183.229.123 (executor 0) (3/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 437.0 (TID 1575)
in 206 ms on 10.183.229.123 (executor 0) (4/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 431.0 (TID 1561)
in 234 ms on 10.183.229.123 (executor 0) (1/5)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 431.0 (TID 1562)
in 239 ms on 10.183.249.197 (executor 7) (2/5)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 431.0 (TID 1563)
in 240 ms on 10.183.246.72 (executor 6) (3/5)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 440.0 (TID 1577)
in 174 ms on 10.183.232.45 (executor 2) (1/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 434.0 (TID 1567)
in 246 ms on 10.183.249.197 (executor 7) (4/6)
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(150), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 431.0 (TID 1559)
in 254 ms on 10.183.233.191 (executor 5) (4/5)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 434.0 (TID 1568)
in 256 ms on 10.183.246.72 (executor 6) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 437.0 (TID 1574)
in 246 ms on 10.183.232.45 (executor 2) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 1.0 in stage 431.0 (TID 1560)
in 257 ms on 10.183.232.45 (executor 2) (5/5)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 431.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 431
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
225.572 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 437,
ShuffleMapStage 440, ShuffleMapStage 434)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:22 INFO TaskSetManager: Finished task 2.0 in stage 440.0 (TID 1578)
in 155 ms on 10.183.233.191 (executor 5) (2/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 437.0 (TID 1570)
in 285 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 437.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 437
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
135.567 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 440,
ShuffleMapStage 434)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO TaskSetManager: Finished task 3.0 in stage 440.0 (TID 1579)
in 161 ms on 10.183.246.72 (executor 6) (3/6)
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(166), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO TaskSetManager: Finished task 5.0 in stage 440.0 (TID 1581)
in 109 ms on 10.183.246.72 (executor 6) (4/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 434.0 (TID 1564)
in 315 ms on 10.183.233.191 (executor 5) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 434.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 434
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
181.840 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set(ShuffleMapStage 440)
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO TaskSetManager: Finished task 4.0 in stage 440.0 (TID 1580)
in 118 ms on 10.183.233.191 (executor 5) (5/6)
25/02/06 06:18:22 INFO TaskSetManager: Finished task 0.0 in stage 440.0 (TID 1576)
in 260 ms on 10.183.249.197 (executor 7) (6/6)
25/02/06 06:18:22 INFO TaskSchedulerImpl: Removed TaskSet 440.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:18:22 INFO DAGScheduler: ShuffleMapStage 440
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
111.119 s
25/02/06 06:18:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:18:22 INFO DAGScheduler: running: Set()
25/02/06 06:18:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:18:22 INFO DAGScheduler: failed: Set()
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(159), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(170), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:22 INFO ShufflePartitionsUtil: For shuffle(152), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(169), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(163), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(153), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(167), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(151), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(174), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 INFO ShufflePartitionsUtil: For shuffle(172), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:18:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:18:23 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(218,1738822296051,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@59ec222b, org.apache.spark.scheduler.StageInfo@6547f567,
org.apache.spark.scheduler.StageInfo@3c6edb7d,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 285.766160445s.
25/02/06 06:18:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:35 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:35 INFO ShufflePartitionsUtil: For shuffle(175), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:35 INFO ShufflePartitionsUtil: For shuffle(154), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:35 INFO ShufflePartitionsUtil: For shuffle(157), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO ShufflePartitionsUtil: For shuffle(173), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO ShufflePartitionsUtil: For shuffle(148), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO ShufflePartitionsUtil: For shuffle(158), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO ShufflePartitionsUtil: For shuffle(176), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO ShufflePartitionsUtil: For shuffle(165), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:18:36 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:36 INFO DAGScheduler: Registering RDD 1551
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 237
25/02/06 06:18:36 INFO DAGScheduler: Got map stage job 240
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:18:36 INFO DAGScheduler: Final stage: ShuffleMapStage 442
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:18:36 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:18:36 INFO DAGScheduler: Missing parents: List()
25/02/06 06:18:36 INFO DAGScheduler: Submitting ShuffleMapStage 442
(MapPartitionsRDD[1551] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:18:47 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:47 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:47 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:47 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:18:47 ERROR RetryingHMSHandler: Retrying HMSHandler after 2000 ms
(attempt 3 of 10) with error: javax.jdo.JDODataStoreException: HikariPool-1 -
Connection is not available, request timed out after 36488ms.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(Nucleus
JDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
NestedThrowablesStackTrace:
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not
available, request timed out after 36488ms.
at
com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
at
org.datanucleus.store.rdbms.ConnectionProviderPriorityList.getConnection(Connection
ProviderPriorityList.java:57)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnecti
on(ConnectionFactoryImpl.java:402)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getXAResour
ce(ConnectionFactoryImpl.java:361)
at
org.datanucleus.store.connection.ConnectionManagerImpl.allocateConnection(Connectio
nManagerImpl.java:316)
at
org.datanucleus.store.connection.AbstractConnectionFactory.getConnection(AbstractCo
nnectionFactory.java:84)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
347)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:
310)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:591)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1855)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:228)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:499)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy127.getDatabase(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStor
e.java:796)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:
105)
at com.sun.proxy.$Proxy129.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClien
t.java:949)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
3)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreCl
ient.java:89)
at com.sun.proxy.$Proxy130.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154)
at
org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$databaseExists$1(HiveClientImpl.scala:452)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$withHiveState$1(HiveClientImpl.scala:349)
at org.apache.spark.sql.hive.client.HiveClientImpl.
$anonfun$retryLocked$1(HiveClientImpl.scala:248)
at
org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.
scala:286)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:24
0)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:
329)
at
org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala
:452)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1(PoolingHiveClient.scala:321)
at org.apache.spark.sql.hive.client.PoolingHiveClient.
$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient
.scala:149)
at
org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient
.scala:320)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$databaseExists$1(HiveExternalCatalog.scala:329)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$2(HiveExternalCatalog.scala:156)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog
.scala:117)
at org.apache.spark.sql.hive.HiveExternalCatalog.
$anonfun$withClient$1(HiveExternalCatalog.scala:155)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressRepor
ter.scala:403)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressR
eporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:
154)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.sc
ala:329)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:93)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:783)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalC
atalogWithListener.scala:53)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(Ex
ternalCatalogWithListener.scala:93)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$5(DriverCorral.scala:510)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
at com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
at com.databricks.unity.HandleImpl.
$anonfun$runWithAndClose$1(UCSHandle.scala:108)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.unity.HandleImpl.runWithAndClose(UCSHandle.scala:107)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4(DriverCorral.scala:510)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$4$adapted(DriverCorral.scala:509)
at scala.util.Using$.resource(Using.scala:269)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$new$3(DriverCorral.scala:509)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperationWithResultTags(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.threading.NamedTimer$
$anon$1.recordOperation(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$2(NamedTimer.scala:104)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at com.databricks.threading.NamedTimer$
$anon$1.withAttributionContext(NamedTimer.scala:95)
at
com.databricks.logging.UsageLogging.disableTracing(UsageLogging.scala:1425)
at com.databricks.logging.UsageLogging.disableTracing$
(UsageLogging.scala:1424)
at com.databricks.threading.NamedTimer$
$anon$1.disableTracing(NamedTimer.scala:95)
at com.databricks.threading.NamedTimer$
$anon$1.$anonfun$run$1(NamedTimer.scala:103)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.util.UntrustedUtils$.tryLog(UntrustedUtils.scala:109)
at com.databricks.threading.NamedTimer$$anon$1.run(NamedTimer.scala:102)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to
address=(host=md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com)(port=3306)
(type=master) : Could not connect to md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-
1.rds.amazonaws.com:3306 : unexpected end of stream, read 0 bytes from 4 (socket
was closed by server)
at
org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(Exceptio
nFactory.java:73)
at
org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.
java:197)
at
org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(Abst
ractConnectProtocol.java:1394)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:635)
at
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:150)
at org.mariadb.jdbc.Driver.connect(Driver.java:89)
at
com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at
com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:341)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:193)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:430)
at com.zaxxer.hikari.pool.HikariPool.access$500(HikariPool.java:64)
at
com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:570)
at
com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:563)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to
md15cf9e1wmjgny.cxg30ia2wqgj.eu-west-1.rds.amazonaws.com:3306 : unexpected end of
stream, read 0 bytes from 4 (socket was closed by server)
at
org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(Exceptio
nFactory.java:73)
at
org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.
java:188)
at
org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createConnection(Abstrac
tConnectProtocol.java:575)
at
org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(Abst
ractConnectProtocol.java:1389)
... 15 more
Caused by: java.io.EOFException: unexpected end of stream, read 0 bytes from 4
(socket was closed by server)
at
org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacketArray(Standar
dPacketInputStream.java:249)
at
org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacket(StandardPack
etInputStream.java:218)
at
org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authenticationHandler(Ab
stractConnectProtocol.java:708)
at
org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createConnection(Abstrac
tConnectProtocol.java:555)
... 16 more
25/02/06 06:18:48 INFO DAGScheduler: Registering RDD 1559
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 238
25/02/06 06:18:48 INFO DAGScheduler: Got map stage job 241
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:18:48 INFO DAGScheduler: Final stage: ShuffleMapStage 443
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:18:48 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:18:48 INFO DAGScheduler: Missing parents: List()
25/02/06 06:18:48 INFO DAGScheduler: Submitting ShuffleMapStage 443
(MapPartitionsRDD[1559] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:18:48 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:18:48 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:48 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:18:48 INFO DAGScheduler: Reorder tasks consumed 117ms, search for
parent partitions consumed 2ms, read map sizes consumed 0ms, get parent deps
consumed 0ms, fetch deps consumed 98ms.
25/02/06 06:18:48 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 442 (MapPartitionsRDD[1551] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:18:48 INFO TaskSchedulerImpl: Adding task set 442.0 with 18 tasks
resource profile 0
25/02/06 06:18:48 INFO TaskSetManager: TaskSet 442.0 using PreferredLocationsV1
25/02/06 06:18:48 WARN FairSchedulableBuilder: A job was submitted with scheduler
pool 5400065285585087590, which has not been configured. This can happen when the
file that pools are read from isn't set, or when that file doesn't contain
5400065285585087590. Created 5400065285585087590 with default configuration
(schedulingMode: FIFO, minShare: 0, weight: 1)
25/02/06 06:18:48 INFO FairSchedulableBuilder: Added task set TaskSet_442.0 tasks
to pool 5400065285585087590
25/02/06 06:18:48 INFO TaskSetManager: Starting task 0.0 in stage 442.0 (TID 1582)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:18:48 INFO DAGScheduler: Registering RDD 1561
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 239
25/02/06 06:18:48 INFO TaskSetManager: Starting task 1.0 in stage 442.0 (TID 1583)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:18:48 INFO DAGScheduler: Got map stage job 242
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:18:48 INFO DAGScheduler: Final stage: ShuffleMapStage 444
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:18:48 INFO TaskSetManager: Starting task 2.0 in stage 442.0 (TID 1584)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:18:48 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:18:48 INFO TaskSetManager: Starting task 3.0 in stage 442.0 (TID 1585)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 4.0 in stage 442.0 (TID 1586)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 5.0 in stage 442.0 (TID 1587)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 6.0 in stage 442.0 (TID 1588)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 7.0 in stage 442.0 (TID 1589)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 8.0 in stage 442.0 (TID 1590)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 9.0 in stage 442.0 (TID 1591)
(10.183.232.45, executor 2, partition 9, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 10.0 in stage 442.0 (TID 1592)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 11.0 in stage 442.0 (TID 1593)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 12.0 in stage 442.0 (TID 1594)
(10.183.249.197, executor 7, partition 12, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 13.0 in stage 442.0 (TID 1595)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 14.0 in stage 442.0 (TID 1596)
(10.183.232.45, executor 2, partition 14, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 15.0 in stage 442.0 (TID 1597)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 16.0 in stage 442.0 (TID 1598)
(10.183.246.72, executor 6, partition 16, PROCESS_LOCAL,
25/02/06 06:18:48 INFO TaskSetManager: Starting task 17.0 in stage 442.0 (TID 1599)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:18:48 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:00 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:19:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:19:00 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:00 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:00 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:00 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO MemoryStore: Block broadcast_92 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:19:00 INFO DAGScheduler: Submitting ShuffleMapStage 444
(MapPartitionsRDD[1561] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:00 INFO DAGScheduler: Registering RDD 1566
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 242
25/02/06 06:19:00 INFO DAGScheduler: Got map stage job 243
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:00 INFO DAGScheduler: Final stage: ShuffleMapStage 445
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:00 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:00 INFO DAGScheduler: Submitting ShuffleMapStage 445
(MapPartitionsRDD[1566] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:00 INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in
memory (estimated size 126.3 KiB, free 20.2 GiB)
25/02/06 06:19:00 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.232.44:37705 (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:19:00 INFO SparkContext: Created broadcast 92 from broadcast at
TaskSetManager.scala:711
25/02/06 06:19:00 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.229.123:38441 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:00 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:00 INFO DAGScheduler: Reorder tasks consumed 4ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:00 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 443 (MapPartitionsRDD[1559] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:00 INFO TaskSchedulerImpl: Adding task set 443.0 with 18 tasks
resource profile 0
25/02/06 06:19:00 INFO TaskSetManager: TaskSet 443.0 using PreferredLocationsV1
25/02/06 06:19:00 INFO FairSchedulableBuilder: Added task set TaskSet_443.0 tasks
to pool 5400065285585087590
25/02/06 06:19:00 INFO TaskSetManager: Starting task 0.0 in stage 443.0 (TID 1600)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:19:00 INFO TaskSetManager: Starting task 1.0 in stage 443.0 (TID 1601)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:19:00 INFO DAGScheduler: Registering RDD 1563
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 240
25/02/06 06:19:00 INFO DAGScheduler: Got map stage job 244
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:00 INFO DAGScheduler: Final stage: ShuffleMapStage 446
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:00 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:00 INFO DAGScheduler: Submitting ShuffleMapStage 446
(MapPartitionsRDD[1563] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:00 INFO DAGScheduler: Registering RDD 1564
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 241
25/02/06 06:19:00 INFO DAGScheduler: Got map stage job 245
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:00 INFO DAGScheduler: Final stage: ShuffleMapStage 447
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:00 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:00 INFO DAGScheduler: Submitting ShuffleMapStage 447
(MapPartitionsRDD[1564] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:00 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:00 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 444 (MapPartitionsRDD[1561] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:00 INFO TaskSchedulerImpl: Adding task set 444.0 with 18 tasks
resource profile 0
25/02/06 06:19:00 INFO TaskSetManager: TaskSet 444.0 using PreferredLocationsV1
25/02/06 06:19:00 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.246.72:43231 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:00 INFO FairSchedulableBuilder: Added task set TaskSet_444.0 tasks
to pool 5400065285585087590
25/02/06 06:19:00 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:00 INFO DAGScheduler: Reorder tasks consumed 6ms, search for parent
partitions consumed 5ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:00 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 445 (MapPartitionsRDD[1566] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:00 INFO TaskSchedulerImpl: Adding task set 445.0 with 18 tasks
resource profile 0
25/02/06 06:19:00 INFO TaskSetManager: TaskSet 445.0 using PreferredLocationsV1
25/02/06 06:19:00 INFO FairSchedulableBuilder: Added task set TaskSet_445.0 tasks
to pool 5400065285585087590
25/02/06 06:19:00 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:00 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:00 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 446 (MapPartitionsRDD[1563] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:00 INFO TaskSchedulerImpl: Adding task set 446.0 with 18 tasks
resource profile 0
25/02/06 06:19:00 INFO TaskSetManager: TaskSet 446.0 using PreferredLocationsV1
25/02/06 06:19:00 INFO FairSchedulableBuilder: Added task set TaskSet_446.0 tasks
to pool 5400065285585087590
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:00 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 1ms.
25/02/06 06:19:00 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 447 (MapPartitionsRDD[1564] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:00 INFO TaskSchedulerImpl: Adding task set 447.0 with 18 tasks
resource profile 0
25/02/06 06:19:00 INFO TaskSetManager: TaskSet 447.0 using PreferredLocationsV1
25/02/06 06:19:00 INFO FairSchedulableBuilder: Added task set TaskSet_447.0 tasks
to pool 5400065285585087590
25/02/06 06:19:00 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.249.197:40023 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:00 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:09 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.232.45:45579 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on
10.183.233.191:42923 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO MemoryStore: Block broadcast_93 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:19:09 INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:19:09 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:19:09 INFO SparkContext: Created broadcast 93 from broadcast at
TaskSetManager.scala:711
25/02/06 06:19:09 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:09 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1653
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 245
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 246
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 448
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 448
(MapPartitionsRDD[1653] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1652
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 244
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 247
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 449
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 449
(MapPartitionsRDD[1652] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1650
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 243
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 248
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 450
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 450
(MapPartitionsRDD[1650] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:09 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:09 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 448 (MapPartitionsRDD[1653] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:09 INFO TaskSchedulerImpl: Adding task set 448.0 with 18 tasks
resource profile 0
25/02/06 06:19:09 INFO TaskSetManager: TaskSet 448.0 using PreferredLocationsV1
25/02/06 06:19:09 INFO FairSchedulableBuilder: Added task set TaskSet_448.0 tasks
to pool 5400065285585087590
25/02/06 06:19:09 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:09 INFO DAGScheduler: Reorder tasks consumed 9ms, search for parent
partitions consumed 8ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:09 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 449 (MapPartitionsRDD[1652] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:09 INFO TaskSchedulerImpl: Adding task set 449.0 with 18 tasks
resource profile 0
25/02/06 06:19:09 INFO TaskSetManager: TaskSet 449.0 using PreferredLocationsV1
25/02/06 06:19:09 INFO FairSchedulableBuilder: Added task set TaskSet_449.0 tasks
to pool 5400065285585087590
25/02/06 06:19:09 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:09 INFO DAGScheduler: Reorder tasks consumed 6ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:09 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 450 (MapPartitionsRDD[1650] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:09 INFO TaskSchedulerImpl: Adding task set 450.0 with 18 tasks
resource profile 0
25/02/06 06:19:09 INFO TaskSetManager: TaskSet 450.0 using PreferredLocationsV1
25/02/06 06:19:09 INFO FairSchedulableBuilder: Added task set TaskSet_450.0 tasks
to pool 5400065285585087590
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1665
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 246
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 249
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 451
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 451
(MapPartitionsRDD[1665] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1672
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 249
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 250
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 452
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 452
(MapPartitionsRDD[1672] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1677
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 251
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 251
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 453
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 453
(MapPartitionsRDD[1677] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1667
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 247
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 252
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 454
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 454
(MapPartitionsRDD[1667] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO DAGScheduler: Registering RDD 1681
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 253
25/02/06 06:19:09 INFO DAGScheduler: Got map stage job 253
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:09 INFO DAGScheduler: Final stage: ShuffleMapStage 455
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:09 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:09 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:09 INFO DAGScheduler: Submitting ShuffleMapStage 455
(MapPartitionsRDD[1681] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:09 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:10 INFO PhotonParquetFileScanExec: Planning scan with bin packing,
max split size: 647541670 bytes, max partition size: 647541670, open cost is
considered as scanning 4194304 bytes.
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1676
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 250
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 254
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 456
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 456
(MapPartitionsRDD[1676] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1682
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 254
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 255
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 457
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 457
(MapPartitionsRDD[1682] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1679
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 255
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 256
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 458
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 458
(MapPartitionsRDD[1679] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1680
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 252
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 258
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 459
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 459
(MapPartitionsRDD[1680] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1670
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 248
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 257
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 460
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 460
(MapPartitionsRDD[1670] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 451 (MapPartitionsRDD[1665] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 451.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 451.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_451.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1689
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 256
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 259
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 461
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 461
(MapPartitionsRDD[1689] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1691
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 257
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 260
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 462
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 462
(MapPartitionsRDD[1691] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1693
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 258
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 261
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 463
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 463
(MapPartitionsRDD[1693] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1696
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 260
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 262
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 464
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 464
(MapPartitionsRDD[1696] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1698
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 259
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 264
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 465
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 465
(MapPartitionsRDD[1698] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1699
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 261
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 263
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 466
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 466
(MapPartitionsRDD[1699] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 452 (MapPartitionsRDD[1672] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 452.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 452.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_452.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 453 (MapPartitionsRDD[1677] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 453.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 453.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_453.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 454 (MapPartitionsRDD[1667] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 454.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 454.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_454.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 455 (MapPartitionsRDD[1681] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 455.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 455.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_455.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1706
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 262
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 265
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 467
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 467
(MapPartitionsRDD[1706] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 456 (MapPartitionsRDD[1676] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 456.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 456.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_456.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 457 (MapPartitionsRDD[1682] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 457.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 457.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_457.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1719
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 263
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 266
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 468
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 468
(MapPartitionsRDD[1719] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 458 (MapPartitionsRDD[1679] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 458.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 458.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_458.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 460 (MapPartitionsRDD[1670] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 460.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 460.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_460.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 1ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 1ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 459 (MapPartitionsRDD[1680] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 459.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 459.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_459.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1730
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 264
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 267
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 469
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 469
(MapPartitionsRDD[1730] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 4ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 461 (MapPartitionsRDD[1689] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 461.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 461.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_461.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1734
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 266
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 268
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 470
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 470
(MapPartitionsRDD[1734] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Registering RDD 1732
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 265
25/02/06 06:19:10 INFO DAGScheduler: Got map stage job 269
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 18 output
partitions
25/02/06 06:19:10 INFO DAGScheduler: Final stage: ShuffleMapStage 471
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:10 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
441)
25/02/06 06:19:10 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:10 INFO DAGScheduler: Submitting ShuffleMapStage 471
(MapPartitionsRDD[1732] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 462 (MapPartitionsRDD[1691] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 462.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 462.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_462.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 463 (MapPartitionsRDD[1693] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 463.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 463.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_463.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 464 (MapPartitionsRDD[1696] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 464.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 464.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_464.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 3ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 466 (MapPartitionsRDD[1699] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 466.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 466.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_466.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 1ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 465 (MapPartitionsRDD[1698] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 465.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 465.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_465.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 468 (MapPartitionsRDD[1719] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 468.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 468.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_468.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 467 (MapPartitionsRDD[1706] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 467.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 467.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_467.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 2ms, search for parent
partitions consumed 1ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 469 (MapPartitionsRDD[1730] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 469.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 469.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_469.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 3ms, search for parent
partitions consumed 2ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 470 (MapPartitionsRDD[1734] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 470.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 470.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_470.0 tasks
to pool 5400065285585087590
25/02/06 06:19:10 INFO DAGScheduler: Task reorder enabled for df_online.show() on
partitions (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17).
25/02/06 06:19:10 INFO DAGScheduler: Reorder tasks consumed 1ms, search for parent
partitions consumed 0ms, read map sizes consumed 0ms, get parent deps consumed 0ms,
fetch deps consumed 0ms.
25/02/06 06:19:10 INFO DAGScheduler: Submitting 18 missing tasks from
ShuffleMapStage 471 (MapPartitionsRDD[1732] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/02/06 06:19:10 INFO TaskSchedulerImpl: Adding task set 471.0 with 18 tasks
resource profile 0
25/02/06 06:19:10 INFO TaskSetManager: TaskSet 471.0 using PreferredLocationsV1
25/02/06 06:19:10 INFO FairSchedulableBuilder: Added task set TaskSet_471.0 tasks
to pool 5400065285585087590
25/02/06 06:19:18 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:18 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:18 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(219,1738822296066,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@209e54f2, org.apache.spark.scheduler.StageInfo@185bc7b0,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@42edf511),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 55.034196424s.
25/02/06 06:19:18 INFO DriverCorral: Metastore health check ok
25/02/06 06:19:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.229.123:55848
25/02/06 06:19:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.246.72:39896
25/02/06 06:19:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.232.45:60256
25/02/06 06:19:18 INFO ProgressReporter$: Reporting progress for running commands:
5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9 [8
occurrences]
25/02/06 06:19:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:19 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097647) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097647)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:19:28 INFO TaskSetManager: Finished task 10.0 in stage 442.0 (TID 1592)
in 40772 ms on 10.183.229.123 (executor 0) (1/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 0.0 in stage 442.0 (TID 1582)
in 40773 ms on 10.183.229.123 (executor 0) (2/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 1.0 in stage 443.0 (TID 1601)
in 28663 ms on 10.183.232.45 (executor 2) (1/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 5.0 in stage 442.0 (TID 1587)
in 40773 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 4.0 in stage 442.0 (TID 1586)
in 40773 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 9.0 in stage 442.0 (TID 1591)
in 40773 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:19:28 INFO TaskSetManager: Finished task 14.0 in stage 442.0 (TID 1596)
in 40773 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:19:28 INFO TaskSetManager: Starting task 2.0 in stage 443.0 (TID 1602)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 3.0 in stage 443.0 (TID 1603)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 4.0 in stage 443.0 (TID 1604)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 5.0 in stage 443.0 (TID 1605)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 6.0 in stage 443.0 (TID 1606)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 7.0 in stage 443.0 (TID 1607)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 8.0 in stage 443.0 (TID 1608)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:19:28 INFO TaskSetManager: Starting task 9.0 in stage 443.0 (TID 1609)
(10.183.232.45, executor 2, partition 9, PROCESS_LOCAL,
25/02/06 06:19:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.233.191:55538
25/02/06 06:19:28 INFO TaskSetManager: Finished task 15.0 in stage 442.0 (TID 1597)
in 40787 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:19:28 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:28 INFO MapOutputTrackerMasterEndpoint: Asked to send map output
locations for shuffle 29 to 10.183.249.197:36834
25/02/06 06:19:29 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:19:29 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:19:29 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:29 INFO TaskSetManager: Starting task 10.0 in stage 443.0 (TID 1610)
(10.183.246.72, executor 6, partition 10, PROCESS_LOCAL,
25/02/06 06:19:29 INFO TaskSetManager: Finished task 16.0 in stage 442.0 (TID 1598)
in 40834 ms on 10.183.246.72 (executor 6) (8/18)
25/02/06 06:19:29 INFO TaskSetManager: Finished task 11.0 in stage 442.0 (TID 1593)
in 40835 ms on 10.183.246.72 (executor 6) (9/18)
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 9906.550984 ms
25/02/06 06:19:29 INFO TaskSetManager: Finished task 6.0 in stage 442.0 (TID 1588)
in 40836 ms on 10.183.246.72 (executor 6) (10/18)
25/02/06 06:19:29 INFO TaskSetManager: Finished task 1.0 in stage 442.0 (TID 1583)
in 40836 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:19:29 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:29 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:29 INFO TaskSetManager: Starting task 11.0 in stage 443.0 (TID 1611)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:19:29 INFO TaskSetManager: Starting task 12.0 in stage 443.0 (TID 1612)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:19:29 INFO TaskSetManager: Starting task 13.0 in stage 443.0 (TID 1613)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:19:29 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 9956.115068 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 9958.656517 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1744
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 267
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 270
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 475
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
474)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 475
(MapPartitionsRDD[1744] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1743
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 268
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 271
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 478
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
477)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 478
(MapPartitionsRDD[1743] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 475 (MapPartitionsRDD[1744] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 475.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 475.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_475.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 73.324214 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 478 (MapPartitionsRDD[1743] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 478.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 478.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_478.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1749
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 269
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 272
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 481
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
480)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 481
(MapPartitionsRDD[1749] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1754
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 270
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 273
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 484
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
483)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 484
(MapPartitionsRDD[1754] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 481 (MapPartitionsRDD[1749] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 481.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 481.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_481.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 484 (MapPartitionsRDD[1754] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 484.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 484.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_484.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 100.051751 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 153.653586 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 93.396079 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1761
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 271
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 274
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 487
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
486)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 10076.886252 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 487
(MapPartitionsRDD[1761] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 104.070986 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 487 (MapPartitionsRDD[1761] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 487.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 487.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_487.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 178.585721 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1779
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 273
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 275
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 490
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
489)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 490
(MapPartitionsRDD[1779] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1777
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 272
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 276
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 493
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
492)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 94.711311 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 493
(MapPartitionsRDD[1777] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 490 (MapPartitionsRDD[1779] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 490.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 490.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_490.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 493 (MapPartitionsRDD[1777] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 493.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 493.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_493.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 208.914059 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1790
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 276
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 277
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 496
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
495)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 496
(MapPartitionsRDD[1790] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1786
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 275
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 278
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 499
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
498)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 219.036482 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 499
(MapPartitionsRDD[1786] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 267.721307 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1780
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 274
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 279
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 502
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
501)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 502
(MapPartitionsRDD[1780] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 117.06081 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 336.335246 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 230.191074 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 132.466844 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 203.536243 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 496 (MapPartitionsRDD[1790] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 496.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 496.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_496.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 297.706228 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1794
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 277
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 280
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 505
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
504)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 293.063886 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 505
(MapPartitionsRDD[1794] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 209.52467 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 270.851795 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 317.753342 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 221.692927 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 198.664624 ms
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 199.473875 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1798
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 278
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 281
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 508
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
507)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 220.795506 ms
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 508
(MapPartitionsRDD[1798] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 499 (MapPartitionsRDD[1786] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 499.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 499.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_499.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 230.908741 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1806
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 280
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 282
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 511
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
510)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 511
(MapPartitionsRDD[1806] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO CodeGenerator: Code generated in 240.235647 ms
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1809
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 281
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 283
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 514
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
513)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 514
(MapPartitionsRDD[1809] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1799
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 279
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 284
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 517
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
516)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 517
(MapPartitionsRDD[1799] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 502 (MapPartitionsRDD[1780] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 502.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 502.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_502.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1838
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 285
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 285
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 520
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
519)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 520
(MapPartitionsRDD[1838] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 505 (MapPartitionsRDD[1794] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 505.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 505.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_505.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1828
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 283
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 286
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 523
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
522)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 523
(MapPartitionsRDD[1828] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1823
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 282
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 287
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 526
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
525)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 526
(MapPartitionsRDD[1823] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1855
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 286
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 288
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 529
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
528)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 529
(MapPartitionsRDD[1855] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1830
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 284
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 289
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 532
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
531)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 532
(MapPartitionsRDD[1830] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1860
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 287
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 290
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 535
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
534)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 535
(MapPartitionsRDD[1860] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1864
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 289
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 291
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 538
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
537)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 538
(MapPartitionsRDD[1864] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 508 (MapPartitionsRDD[1798] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 508.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 508.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_508.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1868
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 291
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 293
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 541
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
540)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 541
(MapPartitionsRDD[1868] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1873
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 293
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 295
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 544
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
543)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 544
(MapPartitionsRDD[1873] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1862
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 288
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 292
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 547
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
546)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 547
(MapPartitionsRDD[1862] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1871
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 292
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 294
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 550
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
549)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 550
(MapPartitionsRDD[1871] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1866
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 290
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 296
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 553
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
552)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 553
(MapPartitionsRDD[1866] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1880
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 294
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 297
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 556
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
555)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 556
(MapPartitionsRDD[1880] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 511 (MapPartitionsRDD[1806] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 511.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 511.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_511.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 514 (MapPartitionsRDD[1809] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 514.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 514.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_514.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1882
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 295
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 298
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 559
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
558)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 559
(MapPartitionsRDD[1882] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Registering RDD 1884
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 296
25/02/06 06:19:29 INFO DAGScheduler: Got map stage job 299
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:29 INFO DAGScheduler: Final stage: ShuffleMapStage 562
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:29 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
561)
25/02/06 06:19:29 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:29 INFO DAGScheduler: Submitting ShuffleMapStage 562
(MapPartitionsRDD[1884] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 517 (MapPartitionsRDD[1799] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 517.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 517.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_517.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 520 (MapPartitionsRDD[1838] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 520.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 520.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_520.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 523 (MapPartitionsRDD[1828] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 523.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 523.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_523.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 526 (MapPartitionsRDD[1823] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 526.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 526.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_526.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 529 (MapPartitionsRDD[1855] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 529.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 529.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_529.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 532 (MapPartitionsRDD[1830] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 532.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 532.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_532.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 535 (MapPartitionsRDD[1860] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 535.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 535.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_535.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 538 (MapPartitionsRDD[1864] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 538.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 538.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_538.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 541 (MapPartitionsRDD[1868] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 541.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 541.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_541.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 544 (MapPartitionsRDD[1873] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 544.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 544.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_544.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 547 (MapPartitionsRDD[1862] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 547.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 547.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_547.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 550 (MapPartitionsRDD[1871] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 550.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 550.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_550.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 553 (MapPartitionsRDD[1866] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 553.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 553.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_553.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 556 (MapPartitionsRDD[1880] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 556.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 556.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_556.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 559 (MapPartitionsRDD[1882] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 559.0 with 5 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 559.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_559.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 562 (MapPartitionsRDD[1884] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 1, 2,
4, 3, 0))
25/02/06 06:19:29 INFO TaskSchedulerImpl: Adding task set 562.0 with 6 tasks
resource profile 0
25/02/06 06:19:29 INFO TaskSetManager: TaskSet 562.0 using PreferredLocationsV1
25/02/06 06:19:29 INFO FairSchedulableBuilder: Added task set TaskSet_562.0 tasks
to pool 5400065285585087590
25/02/06 06:19:29 INFO AsyncEventQueue: Process of event
SparkListenerSQLAdaptiveExecutionUpdate(executionId=32, ...) by listener
SQLAppStatusListener took 10.569375632s.
25/02/06 06:19:38 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:38 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:38 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:38 INFO TaskSetManager: Finished task 13.0 in stage 442.0 (TID 1595)
in 50165 ms on 10.183.233.191 (executor 5) (12/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 0.0 in stage 443.0 (TID 1600)
in 38057 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 3.0 in stage 442.0 (TID 1585)
in 50166 ms on 10.183.233.191 (executor 5) (13/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 8.0 in stage 442.0 (TID 1590)
in 50167 ms on 10.183.233.191 (executor 5) (14/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 13.0 in stage 443.0 (TID 1613)
in 9330 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 11.0 in stage 443.0 (TID 1611)
in 9330 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 12.0 in stage 443.0 (TID 1612)
in 9330 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:19:38 INFO TaskSetManager: Starting task 14.0 in stage 443.0 (TID 1614)
(10.183.233.191, executor 5, partition 14, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 15.0 in stage 443.0 (TID 1615)
(10.183.233.191, executor 5, partition 15, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 16.0 in stage 443.0 (TID 1616)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 17.0 in stage 443.0 (TID 1617)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 0.0 in stage 444.0 (TID 1618)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 1.0 in stage 444.0 (TID 1619)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 2.0 in stage 444.0 (TID 1620)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 3.0 in stage 444.0 (TID 1621)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 4.0 in stage 444.0 (TID 1622)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Finished task 3.0 in stage 443.0 (TID 1603)
in 9396 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 10.0 in stage 443.0 (TID 1610)
in 9340 ms on 10.183.246.72 (executor 6) (7/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 4.0 in stage 443.0 (TID 1604)
in 9397 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 5.0 in stage 443.0 (TID 1605)
in 9397 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:19:38 INFO TaskSetManager: Starting task 5.0 in stage 444.0 (TID 1623)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 6.0 in stage 444.0 (TID 1624)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 7.0 in stage 444.0 (TID 1625)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Finished task 2.0 in stage 443.0 (TID 1602)
in 9399 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 9.0 in stage 443.0 (TID 1609)
in 9413 ms on 10.183.232.45 (executor 2) (11/18)
25/02/06 06:19:38 INFO TaskSetManager: Starting task 8.0 in stage 444.0 (TID 1626)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Finished task 6.0 in stage 443.0 (TID 1606)
in 9416 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 8.0 in stage 443.0 (TID 1608)
in 9416 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 7.0 in stage 443.0 (TID 1607)
in 9416 ms on 10.183.232.45 (executor 2) (14/18)
25/02/06 06:19:38 INFO MemoryStore: Block broadcast_94 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 7.0 in stage 442.0 (TID 1589)
in 50196 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 17.0 in stage 442.0 (TID 1599)
in 50197 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 12.0 in stage 442.0 (TID 1594)
in 50197 ms on 10.183.249.197 (executor 7) (17/18)
25/02/06 06:19:38 INFO TaskSetManager: Finished task 2.0 in stage 442.0 (TID 1584)
in 50198 ms on 10.183.249.197 (executor 7) (18/18)
25/02/06 06:19:38 INFO TaskSchedulerImpl: Removed TaskSet 442.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:19:38 INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:19:38 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:19:38 INFO DAGScheduler: ShuffleMapStage 442
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
61.860 s
25/02/06 06:19:38 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:19:38 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 456,
ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 452,
ShuffleMapStage 444, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 446, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 463, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 457,
ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532,
ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 449, ShuffleMapStage 451, ShuffleMapStage 553, ShuffleMapStage 445,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 447,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 443,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:19:38 INFO DAGScheduler: waiting: Set()
25/02/06 06:19:38 INFO DAGScheduler: failed: Set()
25/02/06 06:19:38 INFO SparkContext: Created broadcast 94 from broadcast at
TaskSetManager.scala:711
25/02/06 06:19:38 INFO TaskSetManager: Starting task 9.0 in stage 444.0 (TID 1627)
(10.183.232.45, executor 2, partition 9, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 10.0 in stage 444.0 (TID 1628)
(10.183.232.45, executor 2, partition 10, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 11.0 in stage 444.0 (TID 1629)
(10.183.232.45, executor 2, partition 11, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 12.0 in stage 444.0 (TID 1630)
(10.183.249.197, executor 7, partition 12, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 13.0 in stage 444.0 (TID 1631)
(10.183.249.197, executor 7, partition 13, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 14.0 in stage 444.0 (TID 1632)
(10.183.249.197, executor 7, partition 14, PROCESS_LOCAL,
25/02/06 06:19:38 INFO TaskSetManager: Starting task 15.0 in stage 444.0 (TID 1633)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:19:38 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:38 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:38 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:38 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(183), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(184), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(179), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(177), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(190), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO ShufflePartitionsUtil: For shuffle(187), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:19:38 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(221,1738822297161,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@8926b02, org.apache.spark.scheduler.StageInfo@5af99726,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@1cd9bd8e),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 9.23197888s.
25/02/06 06:19:50 INFO TaskSetManager: Starting task 16.0 in stage 444.0 (TID 1634)
(10.183.246.72, executor 6, partition 16, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 17.0 in stage 444.0 (TID 1635)
(10.183.246.72, executor 6, partition 17, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 0.0 in stage 445.0 (TID 1636)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 1.0 in stage 445.0 (TID 1637)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 2.0 in stage 445.0 (TID 1638)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 3.0 in stage 445.0 (TID 1639)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 4.0 in stage 445.0 (TID 1640)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 5.0 in stage 445.0 (TID 1641)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Finished task 3.0 in stage 444.0 (TID 1621)
in 12144 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:19:50 INFO MemoryStore: Block broadcast_95 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 2.0 in stage 444.0 (TID 1620)
in 12145 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 0.0 in stage 444.0 (TID 1618)
in 12145 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 4.0 in stage 444.0 (TID 1622)
in 12141 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 7.0 in stage 444.0 (TID 1625)
in 12140 ms on 10.183.229.123 (executor 0) (5/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 1.0 in stage 444.0 (TID 1619)
in 12147 ms on 10.183.246.72 (executor 6) (6/18)
25/02/06 06:19:50 INFO MemoryStore: Block broadcast_95_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:19:50 INFO SparkContext: Created broadcast 95 from broadcast at
TaskSetManager.scala:711
25/02/06 06:19:50 INFO TaskSetManager: Finished task 5.0 in stage 444.0 (TID 1623)
in 12141 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 6.0 in stage 444.0 (TID 1624)
in 12145 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:19:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:50 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:19:50 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 16.0 in stage 443.0 (TID 1616)
in 12155 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.232.44:37705 in memory (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.232.45:45579 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.249.197:40023 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.246.72:43231 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 17.0 in stage 443.0 (TID 1617)
in 12183 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 14.0 in stage 443.0 (TID 1614)
in 12184 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 15.0 in stage 443.0 (TID 1615)
in 12185 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:19:50 INFO TaskSchedulerImpl: Removed TaskSet 443.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:19:50 INFO TaskSetManager: Finished task 11.0 in stage 444.0 (TID 1629)
in 12142 ms on 10.183.232.45 (executor 2) (9/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 9.0 in stage 444.0 (TID 1627)
in 12142 ms on 10.183.232.45 (executor 2) (10/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 10.0 in stage 444.0 (TID 1628)
in 12143 ms on 10.183.232.45 (executor 2) (11/18)
25/02/06 06:19:50 INFO TaskSetManager: Finished task 8.0 in stage 444.0 (TID 1626)
in 12162 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:19:50 INFO DAGScheduler: ShuffleMapStage 443
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
62.485 s
25/02/06 06:19:50 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:19:50 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 456,
ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 452,
ShuffleMapStage 444, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 446, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 463, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 457,
ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532,
ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 449, ShuffleMapStage 451, ShuffleMapStage 553, ShuffleMapStage 445,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 447,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508,
ShuffleMapStage 487)
25/02/06 06:19:50 INFO DAGScheduler: waiting: Set()
25/02/06 06:19:50 INFO DAGScheduler: failed: Set()
25/02/06 06:19:50 INFO TaskSetManager: Starting task 6.0 in stage 445.0 (TID 1642)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 7.0 in stage 445.0 (TID 1643)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 8.0 in stage 445.0 (TID 1644)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 9.0 in stage 445.0 (TID 1645)
(10.183.233.191, executor 5, partition 9, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 10.0 in stage 445.0 (TID 1646)
(10.183.232.45, executor 2, partition 10, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 11.0 in stage 445.0 (TID 1647)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 12.0 in stage 445.0 (TID 1648)
(10.183.232.45, executor 2, partition 12, PROCESS_LOCAL,
25/02/06 06:19:50 INFO TaskSetManager: Starting task 13.0 in stage 445.0 (TID 1649)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.233.191:42923 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Removed broadcast_92_piece0 on
10.183.229.123:38441 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:50 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:19:50 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100802) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100802)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:19:50 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 684741)
25/02/06 06:19:50 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 685541)
25/02/06 06:19:50 WARN BlockManager: Asked to remove block join_relation_684741,
which does not exist
25/02/06 06:19:50 WARN BlockManager: Asked to remove block join_relation_685541,
which does not exist
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 98.00986 ms
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 85.267777 ms
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 113.996403 ms
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 109.413234 ms
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 122.007079 ms
25/02/06 06:19:50 INFO CodeGenerator: Code generated in 82.452736 ms
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1892
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 297
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 300
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 565
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
564)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 565
(MapPartitionsRDD[1892] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:50 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 565 (MapPartitionsRDD[1892] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:50 INFO TaskSchedulerImpl: Adding task set 565.0 with 5 tasks
resource profile 0
25/02/06 06:19:50 INFO TaskSetManager: TaskSet 565.0 using PreferredLocationsV1
25/02/06 06:19:50 INFO FairSchedulableBuilder: Added task set TaskSet_565.0 tasks
to pool 5400065285585087590
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1909
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 301
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 302
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 568
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
567)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 568
(MapPartitionsRDD[1909] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1911
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 299
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 303
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 571
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
570)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 571
(MapPartitionsRDD[1911] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:50 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 568 (MapPartitionsRDD[1909] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:50 INFO TaskSchedulerImpl: Adding task set 568.0 with 5 tasks
resource profile 0
25/02/06 06:19:50 INFO TaskSetManager: TaskSet 568.0 using PreferredLocationsV1
25/02/06 06:19:50 INFO FairSchedulableBuilder: Added task set TaskSet_568.0 tasks
to pool 5400065285585087590
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1914
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 302
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 304
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 574
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
573)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 574
(MapPartitionsRDD[1914] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:50 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 571 (MapPartitionsRDD[1911] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:50 INFO TaskSchedulerImpl: Adding task set 571.0 with 5 tasks
resource profile 0
25/02/06 06:19:50 INFO TaskSetManager: TaskSet 571.0 using PreferredLocationsV1
25/02/06 06:19:50 INFO FairSchedulableBuilder: Added task set TaskSet_571.0 tasks
to pool 5400065285585087590
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1906
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 298
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 301
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 6 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 577
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
576)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 577
(MapPartitionsRDD[1906] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:50 INFO DAGScheduler: Registering RDD 1912
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 300
25/02/06 06:19:50 INFO DAGScheduler: Got map stage job 305
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 5 output
partitions
25/02/06 06:19:50 INFO DAGScheduler: Final stage: ShuffleMapStage 580
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:19:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
579)
25/02/06 06:19:50 INFO DAGScheduler: Missing parents: List()
25/02/06 06:19:50 INFO DAGScheduler: Submitting ShuffleMapStage 580
(MapPartitionsRDD[1912] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:19:51 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 574 (MapPartitionsRDD[1914] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:51 INFO TaskSchedulerImpl: Adding task set 574.0 with 6 tasks
resource profile 0
25/02/06 06:19:51 INFO TaskSetManager: TaskSet 574.0 using PreferredLocationsV1
25/02/06 06:19:51 INFO FairSchedulableBuilder: Added task set TaskSet_574.0 tasks
to pool 5400065285585087590
25/02/06 06:19:51 INFO DAGScheduler: Submitting 6 missing tasks from
ShuffleMapStage 577 (MapPartitionsRDD[1906] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(5, 4, 1,
3, 0, 2))
25/02/06 06:19:51 INFO TaskSchedulerImpl: Adding task set 577.0 with 6 tasks
resource profile 0
25/02/06 06:19:51 INFO TaskSetManager: TaskSet 577.0 using PreferredLocationsV1
25/02/06 06:19:51 INFO FairSchedulableBuilder: Added task set TaskSet_577.0 tasks
to pool 5400065285585087590
25/02/06 06:19:51 INFO DAGScheduler: Submitting 5 missing tasks from
ShuffleMapStage 580 (MapPartitionsRDD[1912] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(4, 0, 1,
2, 3))
25/02/06 06:19:51 INFO TaskSchedulerImpl: Adding task set 580.0 with 5 tasks
resource profile 0
25/02/06 06:19:51 INFO TaskSetManager: TaskSet 580.0 using PreferredLocationsV1
25/02/06 06:19:51 INFO FairSchedulableBuilder: Added task set TaskSet_580.0 tasks
to pool 5400065285585087590
25/02/06 06:19:51 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(222,1738822297175,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@31318679, org.apache.spark.scheduler.StageInfo@2a602126,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@3cb6c3eb),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 12.241261599s.
25/02/06 06:20:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:01 INFO BlockManagerInfo: Removed broadcast_93_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Removed broadcast_93_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 7.0 in stage 445.0 (TID 1643)
in 10965 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 13.0 in stage 445.0 (TID 1649)
in 10965 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 9.0 in stage 445.0 (TID 1645)
in 10966 ms on 10.183.233.191 (executor 5) (3/18)
25/02/06 06:20:01 INFO TaskSetManager: Starting task 14.0 in stage 445.0 (TID 1650)
(10.183.233.191, executor 5, partition 14, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 15.0 in stage 445.0 (TID 1651)
(10.183.233.191, executor 5, partition 15, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 16.0 in stage 445.0 (TID 1652)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 17.0 in stage 445.0 (TID 1653)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Finished task 5.0 in stage 445.0 (TID 1641)
in 11014 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 17.0 in stage 444.0 (TID 1635)
in 11017 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:20:01 INFO TaskSetManager: Starting task 0.0 in stage 446.0 (TID 1654)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 1.0 in stage 446.0 (TID 1655)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 2.0 in stage 446.0 (TID 1656)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 3.0 in stage 446.0 (TID 1657)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 4.0 in stage 446.0 (TID 1658)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 5.0 in stage 446.0 (TID 1659)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 6.0 in stage 446.0 (TID 1660)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 7.0 in stage 446.0 (TID 1661)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 8.0 in stage 446.0 (TID 1662)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 9.0 in stage 446.0 (TID 1663)
(10.183.232.45, executor 2, partition 9, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 10.0 in stage 446.0 (TID 1664)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 11.0 in stage 446.0 (TID 1665)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 12.0 in stage 446.0 (TID 1666)
(10.183.232.45, executor 2, partition 12, PROCESS_LOCAL,
25/02/06 06:20:01 INFO TaskSetManager: Starting task 13.0 in stage 446.0 (TID 1667)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:20:01 INFO DBCEventLoggingListener: Rolling event log;
numTimesRolledOver = 3
25/02/06 06:20:01 INFO DBCEventLoggingListener: Rolled active log file
/databricks/driver/eventlogs/9206972423021441211/eventlog to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-20, size =
48412081
25/02/06 06:20:01 INFO MemoryStore: Block broadcast_96 stored as values in memory
(estimated size 349.9 KiB, free 20.2 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Removed broadcast_93_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Removed broadcast_93_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 16.0 in stage 444.0 (TID 1634)
in 11029 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 11.0 in stage 445.0 (TID 1647)
in 10980 ms on 10.183.233.191 (executor 5) (5/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 2.0 in stage 445.0 (TID 1638)
in 11028 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 3.0 in stage 445.0 (TID 1639)
in 11035 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 4.0 in stage 445.0 (TID 1640)
in 11036 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 15.0 in stage 444.0 (TID 1633)
in 23137 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 8.0 in stage 445.0 (TID 1644)
in 10992 ms on 10.183.232.45 (executor 2) (9/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 12.0 in stage 444.0 (TID 1630)
in 23139 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 6.0 in stage 445.0 (TID 1642)
in 10996 ms on 10.183.232.45 (executor 2) (10/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 12.0 in stage 445.0 (TID 1648)
in 10993 ms on 10.183.232.45 (executor 2) (11/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 14.0 in stage 444.0 (TID 1632)
in 23140 ms on 10.183.249.197 (executor 7) (17/18)
25/02/06 06:20:01 INFO TaskSetManager: Finished task 13.0 in stage 444.0 (TID 1631)
in 23140 ms on 10.183.249.197 (executor 7) (18/18)
25/02/06 06:20:01 INFO TaskSchedulerImpl: Removed TaskSet 444.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:01 INFO TaskSetManager: Finished task 10.0 in stage 445.0 (TID 1646)
in 10994 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:20:01 INFO DAGScheduler: ShuffleMapStage 444
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
61.315 s
25/02/06 06:20:01 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:01 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 471, ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 452, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 446, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 453, ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 449,
ShuffleMapStage 451, ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 445,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 447, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:20:01 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:01 INFO DAGScheduler: failed: Set()
25/02/06 06:20:01 INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:20:01 INFO SparkContext: Created broadcast 96 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:01 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Removed broadcast_93_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:01 INFO DBCEventLoggingListener: Logging events to
eventlogs/9206972423021441211/eventlog
25/02/06 06:20:01 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:01 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 686165)
25/02/06 06:20:01 WARN BlockManager: Asked to remove block join_relation_686165,
which does not exist
25/02/06 06:20:02 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(223,1738822297211,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@6d522c3e, org.apache.spark.scheduler.StageInfo@506d6da3,
org.apache.spark.scheduler.StageInfo@7b2f1f99,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 11.027680553s.
25/02/06 06:20:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:13 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:13 INFO BlockManagerInfo: Removed broadcast_94_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Removed broadcast_94_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Removed broadcast_94_piece0 on
10.183.249.197:40023 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 2.0 in stage 446.0 (TID 1656)
in 12273 ms on 10.183.229.123 (executor 0) (1/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 6.0 in stage 446.0 (TID 1660)
in 12273 ms on 10.183.229.123 (executor 0) (2/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 13.0 in stage 446.0 (TID 1667)
in 12274 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 10.0 in stage 446.0 (TID 1664)
in 12274 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:20:13 INFO TaskSetManager: Starting task 14.0 in stage 446.0 (TID 1668)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 15.0 in stage 446.0 (TID 1669)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 16.0 in stage 446.0 (TID 1670)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 17.0 in stage 446.0 (TID 1671)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 0.0 in stage 447.0 (TID 1672)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 1.0 in stage 447.0 (TID 1673)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 2.0 in stage 447.0 (TID 1674)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 3.0 in stage 447.0 (TID 1675)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:20:13 INFO MemoryStore: Block broadcast_97 stored as values in memory
(estimated size 349.9 KiB, free 20.2 GiB)
25/02/06 06:20:13 INFO TaskSetManager: Starting task 4.0 in stage 447.0 (TID 1676)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 5.0 in stage 447.0 (TID 1677)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 6.0 in stage 447.0 (TID 1678)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 7.0 in stage 447.0 (TID 1679)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:20:13 INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:20:13 INFO SparkContext: Created broadcast 97 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:13 INFO TaskSetManager: Starting task 8.0 in stage 447.0 (TID 1680)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 9.0 in stage 447.0 (TID 1681)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 10.0 in stage 447.0 (TID 1682)
(10.183.232.45, executor 2, partition 10, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 11.0 in stage 447.0 (TID 1683)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 12.0 in stage 447.0 (TID 1684)
(10.183.232.45, executor 2, partition 12, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 13.0 in stage 447.0 (TID 1685)
(10.183.249.197, executor 7, partition 13, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 14.0 in stage 447.0 (TID 1686)
(10.183.232.45, executor 2, partition 14, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Starting task 15.0 in stage 447.0 (TID 1687)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:20:13 INFO TaskSetManager: Finished task 7.0 in stage 446.0 (TID 1661)
in 12287 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 17.0 in stage 445.0 (TID 1653)
in 12290 ms on 10.183.233.191 (executor 5) (13/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 3.0 in stage 446.0 (TID 1657)
in 12288 ms on 10.183.246.72 (executor 6) (6/18)
25/02/06 06:20:13 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 14.0 in stage 445.0 (TID 1650)
in 12291 ms on 10.183.233.191 (executor 5) (14/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 15.0 in stage 445.0 (TID 1651)
in 12292 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 16.0 in stage 445.0 (TID 1652)
in 12292 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 11.0 in stage 446.0 (TID 1665)
in 12290 ms on 10.183.249.197 (executor 7) (7/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 0.0 in stage 446.0 (TID 1654)
in 12290 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 4.0 in stage 446.0 (TID 1658)
in 12290 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 8.0 in stage 446.0 (TID 1662)
in 12290 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 5.0 in stage 446.0 (TID 1659)
in 12291 ms on 10.183.232.45 (executor 2) (11/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 12.0 in stage 446.0 (TID 1666)
in 12291 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 9.0 in stage 446.0 (TID 1663)
in 12291 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 1.0 in stage 446.0 (TID 1655)
in 12292 ms on 10.183.232.45 (executor 2) (14/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 1.0 in stage 445.0 (TID 1637)
in 23309 ms on 10.183.246.72 (executor 6) (17/18)
25/02/06 06:20:13 INFO TaskSetManager: Finished task 0.0 in stage 445.0 (TID 1636)
in 23310 ms on 10.183.246.72 (executor 6) (18/18)
25/02/06 06:20:13 INFO TaskSchedulerImpl: Removed TaskSet 445.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:13 INFO DAGScheduler: ShuffleMapStage 445
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
73.552 s
25/02/06 06:20:13 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:13 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 471, ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 452, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 446, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 453, ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 449,
ShuffleMapStage 451, ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 447,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508,
ShuffleMapStage 487)
25/02/06 06:20:13 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:13 INFO DAGScheduler: failed: Set()
25/02/06 06:20:13 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683433)
25/02/06 06:20:13 WARN BlockManager: Asked to remove block join_relation_683433,
which does not exist
25/02/06 06:20:13 INFO BlockManagerInfo: Removed broadcast_94_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Removed broadcast_94_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:13 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:14 INFO DBCEventLoggingListener: Compressed rolled file
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-20 to
/databricks/driver/eventlogs/9206972423021441211/eventlog-2025-02-06--06-20.gz in
12530ms, size = 5296504
25/02/06 06:20:14 INFO DBCEventLoggingListener: Deleted rolled file
eventlogs/9206972423021441211/eventlog-2025-02-06--06-20
25/02/06 06:20:14 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(225,1738822297304,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@718c228c, org.apache.spark.scheduler.StageInfo@d43c35f,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@3678a721),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 11.700144521s.
25/02/06 06:20:23 INFO TaskSetManager: Finished task 7.0 in stage 447.0 (TID 1679)
in 9265 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:20:23 INFO TaskSetManager: Starting task 16.0 in stage 447.0 (TID 1688)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 17.0 in stage 447.0 (TID 1689)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 0.0 in stage 448.0 (TID 1690)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 1.0 in stage 448.0 (TID 1691)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Finished task 6.0 in stage 447.0 (TID 1678)
in 9267 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:20:23 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:23 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:23 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:23 INFO TaskSetManager: Finished task 4.0 in stage 447.0 (TID 1676)
in 9271 ms on 10.183.233.191 (executor 5) (3/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 5.0 in stage 447.0 (TID 1677)
in 9272 ms on 10.183.233.191 (executor 5) (4/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 12.0 in stage 447.0 (TID 1684)
in 9269 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 10.0 in stage 447.0 (TID 1682)
in 9270 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 3.0 in stage 447.0 (TID 1675)
in 9276 ms on 10.183.246.72 (executor 6) (7/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 8.0 in stage 447.0 (TID 1680)
in 9272 ms on 10.183.232.45 (executor 2) (8/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 1.0 in stage 447.0 (TID 1673)
in 9277 ms on 10.183.246.72 (executor 6) (9/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 14.0 in stage 447.0 (TID 1686)
in 9269 ms on 10.183.232.45 (executor 2) (10/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 2.0 in stage 447.0 (TID 1674)
in 9278 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:20:23 INFO TaskSetManager: Starting task 2.0 in stage 448.0 (TID 1692)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 3.0 in stage 448.0 (TID 1693)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 4.0 in stage 448.0 (TID 1694)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 5.0 in stage 448.0 (TID 1695)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 6.0 in stage 448.0 (TID 1696)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 7.0 in stage 448.0 (TID 1697)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 8.0 in stage 448.0 (TID 1698)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 9.0 in stage 448.0 (TID 1699)
(10.183.246.72, executor 6, partition 9, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 10.0 in stage 448.0 (TID 1700)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 11.0 in stage 448.0 (TID 1701)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 12.0 in stage 448.0 (TID 1702)
(10.183.249.197, executor 7, partition 12, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 13.0 in stage 448.0 (TID 1703)
(10.183.249.197, executor 7, partition 13, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Finished task 0.0 in stage 447.0 (TID 1672)
in 9282 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:20:23 INFO BlockManagerInfo: Removed broadcast_95_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO BlockManagerInfo: Removed broadcast_95_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO BlockManagerInfo: Removed broadcast_95_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:23 INFO BlockManagerInfo: Removed broadcast_95_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 9.0 in stage 447.0 (TID 1681)
in 9277 ms on 10.183.249.197 (executor 7) (13/18)
25/02/06 06:20:23 INFO BlockManagerInfo: Removed broadcast_95_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO MemoryStore: Block broadcast_98 stored as values in memory
(estimated size 349.9 KiB, free 20.2 GiB)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 13.0 in stage 447.0 (TID 1685)
in 9275 ms on 10.183.249.197 (executor 7) (14/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 11.0 in stage 447.0 (TID 1683)
in 9279 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 15.0 in stage 447.0 (TID 1687)
in 9278 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:20:23 INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in
memory (estimated size 126.0 KiB, free 20.2 GiB)
25/02/06 06:20:23 INFO AsyncEventQueue: Process of event
SparkListenerSQLAdaptiveExecutionUpdate(executionId=32, ...) by listener
SQLAppStatusListener took 8.493685168s.
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.232.44:37705 (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:20:23 INFO SparkContext: Created broadcast 98 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:23 INFO TaskSetManager: Starting task 14.0 in stage 448.0 (TID 1704)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 15.0 in stage 448.0 (TID 1705)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 16.0 in stage 448.0 (TID 1706)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:20:23 INFO TaskSetManager: Starting task 17.0 in stage 448.0 (TID 1707)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.232.45:45579 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 16.0 in stage 446.0 (TID 1670)
in 9318 ms on 10.183.229.123 (executor 0) (15/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 15.0 in stage 446.0 (TID 1669)
in 9320 ms on 10.183.229.123 (executor 0) (16/18)
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.233.191:42923 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 14.0 in stage 446.0 (TID 1668)
in 9334 ms on 10.183.229.123 (executor 0) (17/18)
25/02/06 06:20:23 INFO TaskSetManager: Finished task 17.0 in stage 446.0 (TID 1671)
in 9335 ms on 10.183.229.123 (executor 0) (18/18)
25/02/06 06:20:23 INFO TaskSchedulerImpl: Removed TaskSet 446.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:23 INFO DAGScheduler: ShuffleMapStage 446
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
82.815 s
25/02/06 06:20:23 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:23 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 471, ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 452, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571,
ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465,
ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459,
ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 449, ShuffleMapStage 451,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 447, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:20:23 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:23 INFO DAGScheduler: failed: Set()
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.249.197:40023 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.246.72:43231 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:23 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 685873)
25/02/06 06:20:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on
10.183.229.123:38441 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:23 WARN BlockManager: Asked to remove block join_relation_685873,
which does not exist
25/02/06 06:20:23 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:20:23 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:20:32 INFO TaskSetManager: Finished task 7.0 in stage 448.0 (TID 1697)
in 9014 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 9.0 in stage 448.0 (TID 1699)
in 9014 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 17.0 in stage 448.0 (TID 1707)
in 8997 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:20:32 INFO TaskSetManager: Starting task 0.0 in stage 449.0 (TID 1708)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 1.0 in stage 449.0 (TID 1709)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 2.0 in stage 449.0 (TID 1710)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 3.0 in stage 449.0 (TID 1711)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Finished task 8.0 in stage 448.0 (TID 1698)
in 9016 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:20:32 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:32 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:32 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:32 INFO TaskSetManager: Finished task 14.0 in stage 448.0 (TID 1704)
in 8999 ms on 10.183.229.123 (executor 0) (5/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 16.0 in stage 448.0 (TID 1706)
in 8999 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 10.0 in stage 448.0 (TID 1700)
in 9017 ms on 10.183.249.197 (executor 7) (7/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 11.0 in stage 448.0 (TID 1701)
in 9018 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 13.0 in stage 448.0 (TID 1703)
in 9018 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 12.0 in stage 448.0 (TID 1702)
in 9019 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 6.0 in stage 448.0 (TID 1696)
in 9031 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 3.0 in stage 448.0 (TID 1693)
in 9032 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 2.0 in stage 448.0 (TID 1692)
in 9033 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:20:32 INFO TaskSetManager: Starting task 4.0 in stage 449.0 (TID 1712)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 5.0 in stage 449.0 (TID 1713)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 6.0 in stage 449.0 (TID 1714)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 7.0 in stage 449.0 (TID 1715)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 8.0 in stage 449.0 (TID 1716)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 9.0 in stage 449.0 (TID 1717)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 10.0 in stage 449.0 (TID 1718)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 11.0 in stage 449.0 (TID 1719)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Finished task 17.0 in stage 447.0 (TID 1689)
in 9045 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:20:32 INFO TaskSetManager: Finished task 16.0 in stage 447.0 (TID 1688)
in 9046 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:20:32 INFO TaskSchedulerImpl: Removed TaskSet 447.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:32 INFO DAGScheduler: ShuffleMapStage 447
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
91.745 s
25/02/06 06:20:32 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:32 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 471, ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 452, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571,
ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465,
ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459,
ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 449, ShuffleMapStage 451,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:20:32 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:32 INFO DAGScheduler: failed: Set()
25/02/06 06:20:32 INFO TaskSetManager: Finished task 15.0 in stage 448.0 (TID 1705)
in 9021 ms on 10.183.229.123 (executor 0) (14/18)
25/02/06 06:20:32 INFO TaskSetManager: Starting task 12.0 in stage 449.0 (TID 1720)
(10.183.232.45, executor 2, partition 12, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 13.0 in stage 449.0 (TID 1721)
(10.183.232.45, executor 2, partition 13, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 14.0 in stage 449.0 (TID 1722)
(10.183.232.45, executor 2, partition 14, PROCESS_LOCAL,
25/02/06 06:20:32 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 685229)
25/02/06 06:20:32 INFO BlockManagerInfo: Removed broadcast_96_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Removed broadcast_96_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Removed broadcast_96_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Removed broadcast_96_piece0 on
10.183.232.45:45579 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Removed broadcast_96_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:32 WARN BlockManager: Asked to remove block join_relation_685229,
which does not exist
25/02/06 06:20:32 INFO TaskSetManager: Starting task 15.0 in stage 449.0 (TID 1723)
(10.183.232.45, executor 2, partition 15, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Finished task 5.0 in stage 448.0 (TID 1695)
in 9055 ms on 10.183.232.45 (executor 2) (15/18)
25/02/06 06:20:32 INFO TaskSetManager: Starting task 16.0 in stage 449.0 (TID 1724)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Starting task 17.0 in stage 449.0 (TID 1725)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:20:32 INFO TaskSetManager: Finished task 4.0 in stage 448.0 (TID 1694)
in 9060 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:20:32 INFO MemoryStore: Block broadcast_99 stored as values in memory
(estimated size 349.7 KiB, free 20.2 GiB)
25/02/06 06:20:32 INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:32 INFO SparkContext: Created broadcast 99 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:32 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(228,1738822298664,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@35a8b5a0, org.apache.spark.scheduler.StageInfo@252079db,
org.apache.spark.scheduler.StageInfo@36ea0f1b,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 8.929190409s.
25/02/06 06:20:42 INFO TaskSetManager: Finished task 4.0 in stage 449.0 (TID 1712)
in 10239 ms on 10.183.229.123 (executor 0) (1/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 5.0 in stage 449.0 (TID 1713)
in 10239 ms on 10.183.229.123 (executor 0) (2/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 7.0 in stage 449.0 (TID 1715)
in 10240 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 6.0 in stage 449.0 (TID 1714)
in 10240 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 12.0 in stage 449.0 (TID 1720)
in 10235 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 15.0 in stage 449.0 (TID 1723)
in 10236 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:20:42 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:42 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:42 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:42 INFO BlockManagerInfo: Removed broadcast_97_piece0 on
10.183.232.45:45579 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Removed broadcast_97_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Removed broadcast_97_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 13.0 in stage 449.0 (TID 1721)
in 10252 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 10.0 in stage 449.0 (TID 1718)
in 10257 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 11.0 in stage 449.0 (TID 1719)
in 10257 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 8.0 in stage 449.0 (TID 1716)
in 10258 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 9.0 in stage 449.0 (TID 1717)
in 10259 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 1.0 in stage 449.0 (TID 1709)
in 10278 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 3.0 in stage 449.0 (TID 1711)
in 10279 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 2.0 in stage 449.0 (TID 1710)
in 10279 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:20:42 INFO TaskSetManager: Starting task 0.0 in stage 450.0 (TID 1726)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 1.0 in stage 450.0 (TID 1727)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 2.0 in stage 450.0 (TID 1728)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 3.0 in stage 450.0 (TID 1729)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:20:42 INFO AsyncEventQueue: Process of event
SparkListenerSQLAdaptiveExecutionUpdate(executionId=32, ...) by listener
SQLAppStatusListener took 9.489245116s.
25/02/06 06:20:42 INFO BlockManagerInfo: Removed broadcast_97_piece0 on
10.183.233.191:42923 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO MemoryStore: Block broadcast_100 stored as values in memory
(estimated size 349.7 KiB, free 20.2 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Removed broadcast_97_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 1.0 in stage 448.0 (TID 1691)
in 19314 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 0.0 in stage 449.0 (TID 1708)
in 10289 ms on 10.183.246.72 (executor 6) (15/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 14.0 in stage 449.0 (TID 1722)
in 10266 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:20:42 INFO TaskSetManager: Finished task 17.0 in stage 449.0 (TID 1725)
in 10250 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:20:42 INFO TaskSetManager: Starting task 4.0 in stage 450.0 (TID 1730)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 5.0 in stage 450.0 (TID 1731)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 6.0 in stage 450.0 (TID 1732)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 7.0 in stage 450.0 (TID 1733)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 8.0 in stage 450.0 (TID 1734)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 9.0 in stage 450.0 (TID 1735)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 10.0 in stage 450.0 (TID 1736)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 11.0 in stage 450.0 (TID 1737)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 12.0 in stage 450.0 (TID 1738)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 13.0 in stage 450.0 (TID 1739)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 14.0 in stage 450.0 (TID 1740)
(10.183.246.72, executor 6, partition 14, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 15.0 in stage 450.0 (TID 1741)
(10.183.246.72, executor 6, partition 15, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 16.0 in stage 450.0 (TID 1742)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 17.0 in stage 450.0 (TID 1743)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 0.0 in stage 451.0 (TID 1744)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:20:42 INFO TaskSetManager: Starting task 1.0 in stage 451.0 (TID 1745)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:20:42 INFO MemoryStore: Block broadcast_100_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:42 INFO SparkContext: Created broadcast 100 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:42 INFO TaskSetManager: Finished task 16.0 in stage 449.0 (TID 1724)
in 10255 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:20:42 INFO TaskSchedulerImpl: Removed TaskSet 449.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:42 INFO TaskSetManager: Finished task 0.0 in stage 448.0 (TID 1690)
in 19320 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:20:42 INFO TaskSchedulerImpl: Removed TaskSet 448.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:42 INFO DAGScheduler: ShuffleMapStage 449
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
92.612 s
25/02/06 06:20:42 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:42 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 448, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 471, ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 452, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571,
ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465,
ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459,
ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 451, ShuffleMapStage 574,
ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526,
ShuffleMapStage 468, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:20:42 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:42 INFO DAGScheduler: failed: Set()
25/02/06 06:20:42 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682445)
25/02/06 06:20:42 INFO DAGScheduler: ShuffleMapStage 448
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
92.629 s
25/02/06 06:20:42 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:42 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 450, ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 452,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 457,
ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532,
ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 453, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 580, ShuffleMapStage 451, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508,
ShuffleMapStage 487)
25/02/06 06:20:42 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:42 INFO DAGScheduler: failed: Set()
25/02/06 06:20:42 INFO MemoryStore: Block broadcast_101 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:42 WARN BlockManager: Asked to remove block join_relation_682445,
which does not exist
25/02/06 06:20:42 INFO MemoryStore: Block broadcast_101_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:20:42 INFO SparkContext: Created broadcast 101 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:42 INFO ShufflePartitionsUtil: For shuffle(244), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:20:51 INFO TaskSetManager: Finished task 13.0 in stage 450.0 (TID 1739)
in 8856 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 14.0 in stage 450.0 (TID 1740)
in 8855 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 12.0 in stage 450.0 (TID 1738)
in 8857 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:20:51 INFO TaskSetManager: Starting task 2.0 in stage 451.0 (TID 1746)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 3.0 in stage 451.0 (TID 1747)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 4.0 in stage 451.0 (TID 1748)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 5.0 in stage 451.0 (TID 1749)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Finished task 15.0 in stage 450.0 (TID 1741)
in 8857 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 2.0 in stage 450.0 (TID 1728)
in 8875 ms on 10.183.229.123 (executor 0) (5/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 0.0 in stage 450.0 (TID 1726)
in 8875 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 1.0 in stage 450.0 (TID 1727)
in 8875 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:20:51 INFO TaskSetManager: Starting task 6.0 in stage 451.0 (TID 1750)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 7.0 in stage 451.0 (TID 1751)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 8.0 in stage 451.0 (TID 1752)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 9.0 in stage 451.0 (TID 1753)
(10.183.229.123, executor 0, partition 9, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 10.0 in stage 451.0 (TID 1754)
(10.183.233.191, executor 5, partition 10, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 11.0 in stage 451.0 (TID 1755)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 12.0 in stage 451.0 (TID 1756)
(10.183.233.191, executor 5, partition 12, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 13.0 in stage 451.0 (TID 1757)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:20:51 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:51 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:51 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:20:51 INFO TaskSetManager: Finished task 3.0 in stage 450.0 (TID 1729)
in 8883 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:20:51 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 0.0 in stage 451.0 (TID 1744)
in 8872 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:20:51 INFO TaskSetManager: Starting task 14.0 in stage 451.0 (TID 1758)
(10.183.249.197, executor 7, partition 14, PROCESS_LOCAL,
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.229.123:38441 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.246.72:43231 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.232.45:45579 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.249.197:40023 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.249.197:40023 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 17.0 in stage 450.0 (TID 1743)
in 8880 ms on 10.183.233.191 (executor 5) (9/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 16.0 in stage 450.0 (TID 1742)
in 8881 ms on 10.183.233.191 (executor 5) (10/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 1.0 in stage 451.0 (TID 1745)
in 8881 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 8.0 in stage 450.0 (TID 1734)
in 8882 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:20:51 INFO TaskSetManager: Starting task 15.0 in stage 451.0 (TID 1759)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 16.0 in stage 451.0 (TID 1760)
(10.183.249.197, executor 7, partition 16, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 17.0 in stage 451.0 (TID 1761)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 0.0 in stage 452.0 (TID 1762)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 1.0 in stage 452.0 (TID 1763)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Finished task 11.0 in stage 450.0 (TID 1737)
in 8884 ms on 10.183.249.197 (executor 7) (12/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 5.0 in stage 450.0 (TID 1731)
in 8885 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:20:51 INFO TaskSetManager: Starting task 2.0 in stage 452.0 (TID 1764)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Starting task 3.0 in stage 452.0 (TID 1765)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:20:51 INFO TaskSetManager: Finished task 9.0 in stage 450.0 (TID 1735)
in 8885 ms on 10.183.249.197 (executor 7) (14/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 7.0 in stage 450.0 (TID 1733)
in 8887 ms on 10.183.232.45 (executor 2) (15/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 6.0 in stage 450.0 (TID 1732)
in 8888 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 10.0 in stage 450.0 (TID 1736)
in 8887 ms on 10.183.249.197 (executor 7) (17/18)
25/02/06 06:20:51 INFO TaskSetManager: Finished task 4.0 in stage 450.0 (TID 1730)
in 8888 ms on 10.183.232.45 (executor 2) (18/18)
25/02/06 06:20:51 INFO TaskSchedulerImpl: Removed TaskSet 450.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:20:51 INFO DAGScheduler: ShuffleMapStage 450
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
101.482 s
25/02/06 06:20:51 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:20:51 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 452, ShuffleMapStage 517,
ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490,
ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565,
ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511,
ShuffleMapStage 461, ShuffleMapStage 453, ShuffleMapStage 505, ShuffleMapStage 484,
ShuffleMapStage 455, ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580,
ShuffleMapStage 451, ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:20:51 INFO DAGScheduler: waiting: Set()
25/02/06 06:20:51 INFO DAGScheduler: failed: Set()
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.233.191:42923 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_98_piece0 on
10.183.232.44:37705 in memory (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:20:51 INFO MemoryStore: Block broadcast_102 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:20:51 INFO MemoryStore: Block broadcast_102_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:51 INFO BlockManagerInfo: Removed broadcast_99_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:20:51 INFO SparkContext: Created broadcast 102 from broadcast at
TaskSetManager.scala:711
25/02/06 06:20:51 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 5.0 in stage 451.0 (TID 1749)
in 11441 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:21:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:02 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:02 INFO TaskSetManager: Finished task 7.0 in stage 451.0 (TID 1751)
in 11447 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:21:02 INFO TaskSetManager: Starting task 4.0 in stage 452.0 (TID 1766)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 5.0 in stage 452.0 (TID 1767)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 6.0 in stage 452.0 (TID 1768)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 7.0 in stage 452.0 (TID 1769)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 8.0 in stage 452.0 (TID 1770)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 9.0 in stage 452.0 (TID 1771)
(10.183.229.123, executor 0, partition 9, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 10.0 in stage 452.0 (TID 1772)
(10.183.233.191, executor 5, partition 10, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 11.0 in stage 452.0 (TID 1773)
(10.183.229.123, executor 0, partition 11, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 12.0 in stage 452.0 (TID 1774)
(10.183.233.191, executor 5, partition 12, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 13.0 in stage 452.0 (TID 1775)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 14.0 in stage 452.0 (TID 1776)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Finished task 3.0 in stage 451.0 (TID 1747)
in 11473 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 4.0 in stage 451.0 (TID 1748)
in 11474 ms on 10.183.246.72 (executor 6) (6/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 2.0 in stage 451.0 (TID 1746)
in 11475 ms on 10.183.246.72 (executor 6) (7/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 8.0 in stage 451.0 (TID 1752)
in 11467 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Starting task 15.0 in stage 452.0 (TID 1777)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 16.0 in stage 452.0 (TID 1778)
(10.183.249.197, executor 7, partition 16, PROCESS_LOCAL,
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Starting task 17.0 in stage 452.0 (TID 1779)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 0.0 in stage 453.0 (TID 1780)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 1.0 in stage 453.0 (TID 1781)
(10.183.232.45, executor 2, partition 1, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 2.0 in stage 453.0 (TID 1782)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 3.0 in stage 453.0 (TID 1783)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 4.0 in stage 453.0 (TID 1784)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:21:02 INFO TaskSetManager: Starting task 5.0 in stage 453.0 (TID 1785)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:21:02 INFO MemoryStore: Block broadcast_103 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:21:02 INFO MemoryStore: Block broadcast_103_piece0 stored as bytes in
memory (estimated size 126.3 KiB, free 20.2 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 9.0 in stage 451.0 (TID 1753)
in 11482 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on
10.183.232.44:37705 (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:21:02 INFO SparkContext: Created broadcast 103 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:02 INFO TaskSetManager: Finished task 12.0 in stage 451.0 (TID 1756)
in 11479 ms on 10.183.233.191 (executor 5) (10/18)
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.249.197:40023 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 13.0 in stage 451.0 (TID 1757)
in 11492 ms on 10.183.233.191 (executor 5) (11/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 10.0 in stage 451.0 (TID 1754)
in 11493 ms on 10.183.233.191 (executor 5) (12/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 17.0 in stage 451.0 (TID 1761)
in 11481 ms on 10.183.249.197 (executor 7) (13/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 16.0 in stage 451.0 (TID 1760)
in 11482 ms on 10.183.249.197 (executor 7) (14/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 15.0 in stage 451.0 (TID 1759)
in 11483 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 14.0 in stage 451.0 (TID 1758)
in 11498 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 3.0 in stage 452.0 (TID 1765)
in 11490 ms on 10.183.232.45 (executor 2) (1/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 0.0 in stage 452.0 (TID 1762)
in 11494 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 1.0 in stage 452.0 (TID 1763)
in 11496 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 2.0 in stage 452.0 (TID 1764)
in 11497 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 11.0 in stage 451.0 (TID 1755)
in 11516 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO TaskSetManager: Finished task 6.0 in stage 451.0 (TID 1750)
in 11539 ms on 10.183.229.123 (executor 0) (18/18)
25/02/06 06:21:02 INFO TaskSchedulerImpl: Removed TaskSet 451.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO DAGScheduler: ShuffleMapStage 451
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
112.935 s
25/02/06 06:21:02 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:02 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 452, ShuffleMapStage 517,
ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490,
ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565,
ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511,
ShuffleMapStage 461, ShuffleMapStage 453, ShuffleMapStage 505, ShuffleMapStage 484,
ShuffleMapStage 455, ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:21:02 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:02 INFO DAGScheduler: failed: Set()
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on
10.183.232.45:45579 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on
10.183.249.197:40023 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Removed broadcast_100_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:21:02 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on
10.183.233.191:42923 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:02 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 686321)
25/02/06 06:21:02 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682309)
25/02/06 06:21:02 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 684097)
25/02/06 06:21:02 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:02 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:02 WARN BlockManager: Asked to remove block join_relation_686321,
which does not exist
25/02/06 06:21:02 WARN BlockManager: Asked to remove block join_relation_682309,
which does not exist
25/02/06 06:21:02 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(231,1738822299773,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@45d8e748, org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@352d5fe6,
org.apache.spark.scheduler.StageInfo@13650639),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 20.072531402s.
25/02/06 06:21:02 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2097503) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2097503)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:21:02 WARN BlockManager: Asked to remove block join_relation_684097,
which does not exist
25/02/06 06:21:03 INFO CodeGenerator: Code generated in 190.326745 ms
25/02/06 06:21:03 INFO ShufflePartitionsUtil: For shuffle(243), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:21:03 INFO DAGScheduler: Registering RDD 1932
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 303
25/02/06 06:21:03 INFO DAGScheduler: Got map stage job 306
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:21:03 INFO DAGScheduler: Final stage: ShuffleMapStage 591
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:21:03 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
589, ShuffleMapStage 590, ShuffleMapStage 587, ShuffleMapStage 588)
25/02/06 06:21:03 INFO DAGScheduler: Missing parents: List()
25/02/06 06:21:03 INFO DAGScheduler: Submitting ShuffleMapStage 591
(MapPartitionsRDD[1932] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:21:03 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 591 (MapPartitionsRDD[1932] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:21:03 INFO TaskSchedulerImpl: Adding task set 591.0 with 1 tasks
resource profile 0
25/02/06 06:21:03 INFO TaskSetManager: TaskSet 591.0 using PreferredLocationsV1
25/02/06 06:21:03 INFO FairSchedulableBuilder: Added task set TaskSet_591.0 tasks
to pool 5400065285585087590
25/02/06 06:21:12 INFO TaskSetManager: Finished task 14.0 in stage 452.0 (TID 1776)
in 9561 ms on 10.183.229.123 (executor 0) (5/18)
25/02/06 06:21:12 INFO TaskSetManager: Starting task 6.0 in stage 453.0 (TID 1786)
(10.183.229.123, executor 0, partition 6, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 7.0 in stage 453.0 (TID 1787)
(10.183.229.123, executor 0, partition 7, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 8.0 in stage 453.0 (TID 1788)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 9.0 in stage 453.0 (TID 1789)
(10.183.229.123, executor 0, partition 9, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 10.0 in stage 453.0 (TID 1790)
(10.183.232.45, executor 2, partition 10, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 11.0 in stage 453.0 (TID 1791)
(10.183.232.45, executor 2, partition 11, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Finished task 13.0 in stage 452.0 (TID 1775)
in 9564 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 11.0 in stage 452.0 (TID 1773)
in 9564 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 3.0 in stage 453.0 (TID 1783)
in 9547 ms on 10.183.232.45 (executor 2) (1/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 9.0 in stage 452.0 (TID 1771)
in 9570 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 1.0 in stage 453.0 (TID 1781)
in 9553 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:21:12 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:12 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:12 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:12 INFO TaskSetManager: Finished task 4.0 in stage 453.0 (TID 1784)
in 9554 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 2.0 in stage 453.0 (TID 1782)
in 9555 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 17.0 in stage 452.0 (TID 1779)
in 9556 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 16.0 in stage 452.0 (TID 1778)
in 9559 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 15.0 in stage 452.0 (TID 1777)
in 9559 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:21:12 INFO BlockManagerInfo: Removed broadcast_101_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Removed broadcast_101_piece0 on
10.183.233.191:42923 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Removed broadcast_101_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Removed broadcast_101_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on
10.183.229.123:38441 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO TaskSetManager: Starting task 12.0 in stage 453.0 (TID 1792)
(10.183.232.45, executor 2, partition 12, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 13.0 in stage 453.0 (TID 1793)
(10.183.232.45, executor 2, partition 13, PROCESS_LOCAL,
25/02/06 06:21:12 INFO BlockManagerInfo: Removed broadcast_101_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:21:12 INFO TaskSetManager: Starting task 14.0 in stage 453.0 (TID 1794)
(10.183.249.197, executor 7, partition 14, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 15.0 in stage 453.0 (TID 1795)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 16.0 in stage 453.0 (TID 1796)
(10.183.249.197, executor 7, partition 16, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 17.0 in stage 453.0 (TID 1797)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 0.0 in stage 454.0 (TID 1798)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 1.0 in stage 454.0 (TID 1799)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 2.0 in stage 454.0 (TID 1800)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 3.0 in stage 454.0 (TID 1801)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 4.0 in stage 454.0 (TID 1802)
(10.183.233.191, executor 5, partition 4, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 5.0 in stage 454.0 (TID 1803)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:21:12 INFO MemoryStore: Block broadcast_104 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:21:12 INFO TaskSetManager: Starting task 6.0 in stage 454.0 (TID 1804)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:21:12 INFO TaskSetManager: Starting task 7.0 in stage 454.0 (TID 1805)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:21:12 INFO MemoryStore: Block broadcast_104_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 4.0 in stage 452.0 (TID 1766)
in 9604 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:21:12 INFO SparkContext: Created broadcast 104 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:12 INFO TaskSetManager: Finished task 6.0 in stage 452.0 (TID 1768)
in 9605 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 10.0 in stage 452.0 (TID 1772)
in 9605 ms on 10.183.233.191 (executor 5) (14/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 12.0 in stage 452.0 (TID 1774)
in 9606 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 5.0 in stage 453.0 (TID 1785)
in 9589 ms on 10.183.233.191 (executor 5) (5/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 8.0 in stage 452.0 (TID 1770)
in 9607 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 7.0 in stage 452.0 (TID 1769)
in 9607 ms on 10.183.246.72 (executor 6) (17/18)
25/02/06 06:21:12 INFO TaskSetManager: Finished task 5.0 in stage 452.0 (TID 1767)
in 9607 ms on 10.183.246.72 (executor 6) (18/18)
25/02/06 06:21:12 INFO TaskSchedulerImpl: Removed TaskSet 452.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:12 INFO TaskSetManager: Finished task 0.0 in stage 453.0 (TID 1780)
in 9592 ms on 10.183.249.197 (executor 7) (6/18)
25/02/06 06:21:12 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:12 INFO DAGScheduler: ShuffleMapStage 452
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
122.412 s
25/02/06 06:21:12 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:12 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496,
ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 453, ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 574,
ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526,
ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:21:12 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:12 INFO DAGScheduler: failed: Set()
25/02/06 06:21:22 INFO TaskSetManager: Finished task 2.0 in stage 454.0 (TID 1800)
in 9866 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 3.0 in stage 454.0 (TID 1801)
in 9866 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 0.0 in stage 454.0 (TID 1798)
in 9867 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 1.0 in stage 454.0 (TID 1799)
in 9868 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:21:22 INFO TaskSetManager: Starting task 8.0 in stage 454.0 (TID 1806)
(10.183.246.72, executor 6, partition 8, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 9.0 in stage 454.0 (TID 1807)
(10.183.246.72, executor 6, partition 9, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 10.0 in stage 454.0 (TID 1808)
(10.183.246.72, executor 6, partition 10, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 11.0 in stage 454.0 (TID 1809)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 12.0 in stage 454.0 (TID 1810)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 13.0 in stage 454.0 (TID 1811)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 14.0 in stage 454.0 (TID 1812)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 15.0 in stage 454.0 (TID 1813)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 16.0 in stage 454.0 (TID 1814)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 17.0 in stage 454.0 (TID 1815)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 0.0 in stage 455.0 (TID 1816)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 1.0 in stage 455.0 (TID 1817)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Finished task 6.0 in stage 453.0 (TID 1786)
in 9914 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 9.0 in stage 453.0 (TID 1789)
in 9914 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:21:22 INFO MemoryStore: Block broadcast_105 stored as values in memory
(estimated size 340.8 KiB, free 20.2 GiB)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 7.0 in stage 453.0 (TID 1787)
in 9915 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 8.0 in stage 453.0 (TID 1788)
in 9915 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 6.0 in stage 454.0 (TID 1804)
in 9878 ms on 10.183.233.191 (executor 5) (5/18)
25/02/06 06:21:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:22 INFO TaskSetManager: Finished task 7.0 in stage 454.0 (TID 1805)
in 9879 ms on 10.183.233.191 (executor 5) (6/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 4.0 in stage 454.0 (TID 1802)
in 9879 ms on 10.183.233.191 (executor 5) (7/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 5.0 in stage 454.0 (TID 1803)
in 9880 ms on 10.183.233.191 (executor 5) (8/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 15.0 in stage 453.0 (TID 1795)
in 9882 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 17.0 in stage 453.0 (TID 1797)
in 9883 ms on 10.183.249.197 (executor 7) (12/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 14.0 in stage 453.0 (TID 1794)
in 9889 ms on 10.183.249.197 (executor 7) (13/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 16.0 in stage 453.0 (TID 1796)
in 9884 ms on 10.183.249.197 (executor 7) (14/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 11.0 in stage 453.0 (TID 1791)
in 9920 ms on 10.183.232.45 (executor 2) (15/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 13.0 in stage 453.0 (TID 1793)
in 9892 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 10.0 in stage 453.0 (TID 1790)
in 9921 ms on 10.183.232.45 (executor 2) (17/18)
25/02/06 06:21:22 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.229.123:38441 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO TaskSetManager: Finished task 12.0 in stage 453.0 (TID 1792)
in 9896 ms on 10.183.232.45 (executor 2) (18/18)
25/02/06 06:21:22 INFO TaskSchedulerImpl: Removed TaskSet 453.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:22 INFO DAGScheduler: ShuffleMapStage 453
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
132.258 s
25/02/06 06:21:22 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:22 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 454, ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535,
ShuffleMapStage 456, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496,
ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:21:22 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:22 INFO DAGScheduler: failed: Set()
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO MemoryStore: Block broadcast_105_piece0 stored as bytes in
memory (estimated size 122.3 KiB, free 20.2 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.232.44:37705 (size: 122.3 KiB, free: 20.2 GiB)
25/02/06 06:21:22 INFO SparkContext: Created broadcast 105 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:22 INFO TaskSetManager: Starting task 2.0 in stage 455.0 (TID 1818)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 3.0 in stage 455.0 (TID 1819)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 4.0 in stage 455.0 (TID 1820)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 5.0 in stage 455.0 (TID 1821)
(10.183.249.197, executor 7, partition 5, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 6.0 in stage 455.0 (TID 1822)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 7.0 in stage 455.0 (TID 1823)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 8.0 in stage 455.0 (TID 1824)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:21:22 INFO TaskSetManager: Starting task 9.0 in stage 455.0 (TID 1825)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Removed broadcast_102_piece0 on
10.183.249.197:40023 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.249.197:40023 (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.233.191:42923 (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.232.45:45579 (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:22 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(232,1738822301674,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@77f10ac4, org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@28fca9b3,
org.apache.spark.scheduler.StageInfo@1393365b),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 18.799245963s.
25/02/06 06:21:22 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682793)
25/02/06 06:21:22 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683921)
25/02/06 06:21:22 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:22 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:22 WARN BlockManager: Asked to remove block join_relation_682793,
which does not exist
25/02/06 06:21:22 WARN BlockManager: Asked to remove block join_relation_683921,
which does not exist
25/02/06 06:21:22 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099538) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099538)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:21:22 INFO CodeGenerator: Code generated in 115.906964 ms
25/02/06 06:21:22 INFO DAGScheduler: Registering RDD 1950
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 304
25/02/06 06:21:22 INFO DAGScheduler: Got map stage job 307
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:21:22 INFO DAGScheduler: Final stage: ShuffleMapStage 602
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:21:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
601, ShuffleMapStage 598, ShuffleMapStage 599, ShuffleMapStage 600)
25/02/06 06:21:22 INFO DAGScheduler: Missing parents: List()
25/02/06 06:21:22 INFO DAGScheduler: Submitting ShuffleMapStage 602
(MapPartitionsRDD[1950] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:21:22 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 602 (MapPartitionsRDD[1950] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:21:22 INFO TaskSchedulerImpl: Adding task set 602.0 with 1 tasks
resource profile 0
25/02/06 06:21:22 INFO TaskSetManager: TaskSet 602.0 using PreferredLocationsV1
25/02/06 06:21:22 INFO FairSchedulableBuilder: Added task set TaskSet_602.0 tasks
to pool 5400065285585087590
25/02/06 06:21:33 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:33 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:33 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:33 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:21:33 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:21:33 INFO TaskSetManager: Starting task 10.0 in stage 455.0 (TID 1826)
(10.183.246.72, executor 6, partition 10, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 11.0 in stage 455.0 (TID 1827)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 12.0 in stage 455.0 (TID 1828)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Finished task 12.0 in stage 454.0 (TID 1810)
in 10863 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 11.0 in stage 454.0 (TID 1809)
in 10870 ms on 10.183.246.72 (executor 6) (10/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 9.0 in stage 454.0 (TID 1807)
in 10874 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 8.0 in stage 454.0 (TID 1806)
in 10874 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 10.0 in stage 454.0 (TID 1808)
in 10875 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:21:33 INFO TaskSetManager: Starting task 13.0 in stage 455.0 (TID 1829)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 14.0 in stage 455.0 (TID 1830)
(10.183.246.72, executor 6, partition 14, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Finished task 15.0 in stage 454.0 (TID 1813)
in 10903 ms on 10.183.229.123 (executor 0) (14/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 14.0 in stage 454.0 (TID 1812)
in 10904 ms on 10.183.229.123 (executor 0) (15/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 9.0 in stage 455.0 (TID 1825)
in 10881 ms on 10.183.249.197 (executor 7) (1/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 3.0 in stage 455.0 (TID 1819)
in 10881 ms on 10.183.249.197 (executor 7) (2/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 5.0 in stage 455.0 (TID 1821)
in 10881 ms on 10.183.249.197 (executor 7) (3/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 7.0 in stage 455.0 (TID 1823)
in 10882 ms on 10.183.249.197 (executor 7) (4/18)
25/02/06 06:21:33 INFO HiveMetaStore: 1: get_database: default
25/02/06 06:21:33 INFO TaskSetManager: Finished task 2.0 in stage 455.0 (TID 1818)
in 10882 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:21:33 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database:
default
25/02/06 06:21:33 INFO TaskSetManager: Finished task 16.0 in stage 454.0 (TID 1814)
in 10905 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 4.0 in stage 455.0 (TID 1820)
in 10883 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 8.0 in stage 455.0 (TID 1824)
in 10884 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 6.0 in stage 455.0 (TID 1822)
in 10885 ms on 10.183.232.45 (executor 2) (8/18)
25/02/06 06:21:33 INFO BlockManagerInfo: Removed broadcast_103_piece0 on
10.183.232.45:45579 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Removed broadcast_103_piece0 on
10.183.233.191:42923 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Removed broadcast_103_piece0 on
10.183.249.197:40023 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Removed broadcast_103_piece0 on
10.183.229.123:38441 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 13.0 in stage 454.0 (TID 1811)
in 10912 ms on 10.183.229.123 (executor 0) (17/18)
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.229.123:38441 (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Removed broadcast_103_piece0 on
10.183.232.44:37705 in memory (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:21:33 INFO TaskSetManager: Finished task 17.0 in stage 454.0 (TID 1815)
in 10930 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:21:33 INFO TaskSchedulerImpl: Removed TaskSet 454.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:33 INFO DAGScheduler: ShuffleMapStage 454
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
143.157 s
25/02/06 06:21:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:33 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 456,
ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 602, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496,
ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 455, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:21:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:33 INFO DAGScheduler: failed: Set()
25/02/06 06:21:33 INFO TaskSetManager: Starting task 15.0 in stage 455.0 (TID 1831)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 16.0 in stage 455.0 (TID 1832)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 17.0 in stage 455.0 (TID 1833)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 0.0 in stage 456.0 (TID 1834)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 1.0 in stage 456.0 (TID 1835)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 2.0 in stage 456.0 (TID 1836)
(10.183.249.197, executor 7, partition 2, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 3.0 in stage 456.0 (TID 1837)
(10.183.249.197, executor 7, partition 3, PROCESS_LOCAL,
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on
10.183.246.72:43231 (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO TaskSetManager: Starting task 4.0 in stage 456.0 (TID 1838)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 5.0 in stage 456.0 (TID 1839)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 6.0 in stage 456.0 (TID 1840)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 7.0 in stage 456.0 (TID 1841)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 8.0 in stage 456.0 (TID 1842)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:21:33 INFO TaskSetManager: Starting task 9.0 in stage 456.0 (TID 1843)
(10.183.233.191, executor 5, partition 9, PROCESS_LOCAL,
25/02/06 06:21:33 INFO MemoryStore: Block broadcast_106 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:21:33 INFO MemoryStore: Block broadcast_106_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:21:33 INFO SparkContext: Created broadcast 106 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:33 INFO DriverCorral: DBFS health check ok
25/02/06 06:21:33 INFO ShufflePartitionsUtil: For shuffle(249), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:21:33 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(233,1738822301702,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@6cf12039, org.apache.spark.scheduler.StageInfo@1329b049,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@5e5522ae),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 11.002588574s.
25/02/06 06:21:43 INFO TaskSetManager: Starting task 10.0 in stage 456.0 (TID 1844)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 11.0 in stage 456.0 (TID 1845)
(10.183.229.123, executor 0, partition 11, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 12.0 in stage 456.0 (TID 1846)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:21:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:43 INFO TaskSetManager: Starting task 13.0 in stage 456.0 (TID 1847)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:21:43 INFO BlockManagerInfo: Removed broadcast_104_piece0 on
10.183.233.191:42923 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO BlockManagerInfo: Removed broadcast_104_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO BlockManagerInfo: Removed broadcast_104_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 12.0 in stage 455.0 (TID 1828)
in 10768 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 13.0 in stage 455.0 (TID 1829)
in 10759 ms on 10.183.246.72 (executor 6) (10/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 14.0 in stage 455.0 (TID 1830)
in 10760 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 10.0 in stage 455.0 (TID 1826)
in 10771 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 15.0 in stage 455.0 (TID 1831)
in 10695 ms on 10.183.229.123 (executor 0) (13/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 17.0 in stage 455.0 (TID 1833)
in 10695 ms on 10.183.229.123 (executor 0) (14/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 16.0 in stage 455.0 (TID 1832)
in 10695 ms on 10.183.229.123 (executor 0) (15/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 11.0 in stage 455.0 (TID 1827)
in 10781 ms on 10.183.246.72 (executor 6) (16/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 0.0 in stage 455.0 (TID 1816)
in 21643 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 1.0 in stage 455.0 (TID 1817)
in 21643 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:21:43 INFO TaskSchedulerImpl: Removed TaskSet 455.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:43 INFO TaskSetManager: Finished task 9.0 in stage 456.0 (TID 1843)
in 10704 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 5.0 in stage 456.0 (TID 1839)
in 10705 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 4.0 in stage 456.0 (TID 1838)
in 10706 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 6.0 in stage 456.0 (TID 1840)
in 10706 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 7.0 in stage 456.0 (TID 1841)
in 10707 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:21:43 INFO DAGScheduler: ShuffleMapStage 455
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
153.849 s
25/02/06 06:21:43 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:43 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 456,
ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 602, ShuffleMapStage 471,
ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496,
ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469,
ShuffleMapStage 571, ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 465, ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 580, ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508,
ShuffleMapStage 487)
25/02/06 06:21:43 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:43 INFO DAGScheduler: failed: Set()
25/02/06 06:21:43 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO TaskSetManager: Starting task 14.0 in stage 456.0 (TID 1848)
(10.183.246.72, executor 6, partition 14, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 15.0 in stage 456.0 (TID 1849)
(10.183.246.72, executor 6, partition 15, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 16.0 in stage 456.0 (TID 1850)
(10.183.246.72, executor 6, partition 16, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 17.0 in stage 456.0 (TID 1851)
(10.183.246.72, executor 6, partition 17, PROCESS_LOCAL,
25/02/06 06:21:43 INFO BlockManagerInfo: Removed broadcast_104_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:21:43 INFO TaskSetManager: Starting task 0.0 in stage 457.0 (TID 1852)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 1.0 in stage 457.0 (TID 1853)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 2.0 in stage 457.0 (TID 1854)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 3.0 in stage 457.0 (TID 1855)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 4.0 in stage 457.0 (TID 1856)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 5.0 in stage 457.0 (TID 1857)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 6.0 in stage 457.0 (TID 1858)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:21:43 INFO TaskSetManager: Starting task 7.0 in stage 457.0 (TID 1859)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:21:43 INFO MemoryStore: Block broadcast_107 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:21:43 INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in
memory (estimated size 126.3 KiB, free 20.2 GiB)
25/02/06 06:21:43 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on
10.183.232.44:37705 (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:21:43 INFO SparkContext: Created broadcast 107 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:43 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on
10.183.233.191:42923 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on
10.183.232.45:45579 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO TaskSetManager: Finished task 8.0 in stage 456.0 (TID 1842)
in 10773 ms on 10.183.233.191 (executor 5) (6/18)
25/02/06 06:21:43 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:43 INFO DriverCorral: Metastore health check ok
25/02/06 06:21:44 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682929)
25/02/06 06:21:44 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682077)
25/02/06 06:21:44 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:44 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:21:54 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:54 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:54 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:21:54 WARN BlockManager: Asked to remove block join_relation_682929,
which does not exist
25/02/06 06:21:54 WARN BlockManager: Asked to remove block join_relation_682077,
which does not exist
25/02/06 06:21:54 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098521) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098521)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.233.191:42923 in memory (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.229.123:38441 in memory (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 10.0 in stage 456.0 (TID 1844)
in 10501 ms on 10.183.229.123 (executor 0) (7/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 12.0 in stage 456.0 (TID 1846)
in 10502 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 16.0 in stage 456.0 (TID 1850)
in 10452 ms on 10.183.246.72 (executor 6) (9/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 13.0 in stage 456.0 (TID 1847)
in 10493 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 15.0 in stage 456.0 (TID 1849)
in 10453 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 17.0 in stage 456.0 (TID 1851)
in 10454 ms on 10.183.246.72 (executor 6) (12/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 1.0 in stage 457.0 (TID 1853)
in 10431 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 0.0 in stage 457.0 (TID 1852)
in 10432 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 11.0 in stage 456.0 (TID 1845)
in 10505 ms on 10.183.229.123 (executor 0) (13/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 14.0 in stage 456.0 (TID 1848)
in 10455 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 3.0 in stage 457.0 (TID 1855)
in 10433 ms on 10.183.233.191 (executor 5) (3/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 7.0 in stage 457.0 (TID 1859)
in 10436 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:21:54 INFO TaskSetManager: Starting task 8.0 in stage 457.0 (TID 1860)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 9.0 in stage 457.0 (TID 1861)
(10.183.229.123, executor 0, partition 9, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 10.0 in stage 457.0 (TID 1862)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 11.0 in stage 457.0 (TID 1863)
(10.183.229.123, executor 0, partition 11, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 12.0 in stage 457.0 (TID 1864)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 13.0 in stage 457.0 (TID 1865)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 14.0 in stage 457.0 (TID 1866)
(10.183.246.72, executor 6, partition 14, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 15.0 in stage 457.0 (TID 1867)
(10.183.246.72, executor 6, partition 15, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Finished task 5.0 in stage 457.0 (TID 1857)
in 10442 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 3.0 in stage 456.0 (TID 1837)
in 21187 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 0.0 in stage 456.0 (TID 1834)
in 21188 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 1.0 in stage 456.0 (TID 1835)
in 21189 ms on 10.183.249.197 (executor 7) (17/18)
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.232.45:45579 in memory (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.249.197:40023 in memory (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 4.0 in stage 457.0 (TID 1856)
in 10447 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:21:54 INFO TaskSetManager: Starting task 16.0 in stage 457.0 (TID 1868)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 17.0 in stage 457.0 (TID 1869)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 0.0 in stage 458.0 (TID 1870)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 1.0 in stage 458.0 (TID 1871)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 2.0 in stage 458.0 (TID 1872)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 3.0 in stage 458.0 (TID 1873)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 4.0 in stage 458.0 (TID 1874)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 5.0 in stage 458.0 (TID 1875)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 6.0 in stage 458.0 (TID 1876)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 7.0 in stage 458.0 (TID 1877)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 8.0 in stage 458.0 (TID 1878)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:21:54 INFO TaskSetManager: Starting task 9.0 in stage 458.0 (TID 1879)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:21:54 INFO MemoryStore: Block broadcast_108 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 6.0 in stage 457.0 (TID 1858)
in 10451 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:21:54 INFO TaskSetManager: Finished task 2.0 in stage 456.0 (TID 1836)
in 21194 ms on 10.183.249.197 (executor 7) (18/18)
25/02/06 06:21:54 INFO TaskSchedulerImpl: Removed TaskSet 456.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:21:54 INFO DAGScheduler: ShuffleMapStage 456
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
164.305 s
25/02/06 06:21:54 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:21:54 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550,
ShuffleMapStage 529, ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523,
ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571,
ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465,
ShuffleMapStage 457, ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459,
ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 461, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:21:54 INFO DAGScheduler: waiting: Set()
25/02/06 06:21:54 INFO DAGScheduler: failed: Set()
25/02/06 06:21:54 INFO MemoryStore: Block broadcast_108_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:21:54 INFO SparkContext: Created broadcast 108 from broadcast at
TaskSetManager.scala:711
25/02/06 06:21:54 INFO TaskSetManager: Finished task 2.0 in stage 457.0 (TID 1854)
in 10466 ms on 10.183.233.191 (executor 5) (8/18)
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.232.44:37705 in memory (size: 122.3 KiB, free: 20.2 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on
10.183.229.123:38441 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Removed broadcast_105_piece0 on
10.183.246.72:43231 in memory (size: 122.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on
10.183.246.72:43231 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:21:54 INFO CommChannelWebSocket: [session: 2129319033]
onWebSocketConnect with headers: {Accept-Encoding=[gzip], Cache-Control=[no-cache],
Connection=[Upgrade], Db-Outgoing-Buffer-Throttler-Burst=[60000000], Db-Outgoing-
Buffer-Throttler-Steady-Rate=[6000000], Db-Outgoing-Buffer-Throttler-Warning-
Interval-Sec=[60], Dnc-Connection-Safe-
Flags=[enableLSPGoToDefinitionOnModule_v2=true;enableWSFSPythonModulePeek=false;jed
iAutoImportModulesList=asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,
urllib3;], Host=[10.183.232.44:6062], Pragma=[no-cache], Proxied=[true], Proxied-
Host=[10.183.232.44:6062], Sec-WebSocket-Key=[sepVV05tYCPev07Ymp5v0Q==], Sec-
WebSocket-Version=[13], Upgrade=[websocket], User-Agent=[Jetty/9.4.51.v20230217],
X-Forwarded-For=[10.2.113.226], X-Forwarded-Proto=[https]}
25/02/06 06:21:54 INFO OutgoingDirectNotebookMessageBuffer: [session: 2129319033]
Start MessageSendTask
25/02/06 06:21:54 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(234,1738822301704,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@54fee6d1, org.apache.spark.scheduler.StageInfo@316be49f,
org.apache.spark.scheduler.StageInfo@6f1ea103,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 21.03972572s.
25/02/06 06:21:54 INFO CodeGenerator: Code generated in 142.947994 ms
25/02/06 06:21:54 INFO DAGScheduler: Registering RDD 1968
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 305
25/02/06 06:21:54 INFO DAGScheduler: Got map stage job 308
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:21:54 INFO DAGScheduler: Final stage: ShuffleMapStage 613
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:21:54 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
610, ShuffleMapStage 611, ShuffleMapStage 612, ShuffleMapStage 609)
25/02/06 06:21:54 INFO DAGScheduler: Missing parents: List()
25/02/06 06:21:54 INFO DAGScheduler: Submitting ShuffleMapStage 613
(MapPartitionsRDD[1968] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:21:54 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 613 (MapPartitionsRDD[1968] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:21:54 INFO TaskSchedulerImpl: Adding task set 613.0 with 1 tasks
resource profile 0
25/02/06 06:21:54 INFO TaskSetManager: TaskSet 613.0 using PreferredLocationsV1
25/02/06 06:21:54 INFO FairSchedulableBuilder: Added task set TaskSet_613.0 tasks
to pool 5400065285585087590
25/02/06 06:21:54 INFO ShufflePartitionsUtil: For shuffle(253), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:21:54 INFO ShufflePartitionsUtil: For shuffle(247), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:22:06 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:06 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:06 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:06 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.232.45:45579 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.233.191:42923 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO TaskSetManager: Starting task 10.0 in stage 458.0 (TID 1880)
(10.183.246.72, executor 6, partition 10, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 11.0 in stage 458.0 (TID 1881)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 12.0 in stage 458.0 (TID 1882)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 13.0 in stage 458.0 (TID 1883)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 14.0 in stage 458.0 (TID 1884)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 15.0 in stage 458.0 (TID 1885)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 16.0 in stage 458.0 (TID 1886)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 17.0 in stage 458.0 (TID 1887)
(10.183.232.45, executor 2, partition 17, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 0.0 in stage 460.0 (TID 1888)
(10.183.232.45, executor 2, partition 0, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 1.0 in stage 460.0 (TID 1889)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 2.0 in stage 460.0 (TID 1890)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 3.0 in stage 460.0 (TID 1891)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 4.0 in stage 460.0 (TID 1892)
(10.183.249.197, executor 7, partition 4, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 5.0 in stage 460.0 (TID 1893)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 6.0 in stage 460.0 (TID 1894)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 7.0 in stage 460.0 (TID 1895)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Starting task 8.0 in stage 460.0 (TID 1896)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:22:06 INFO MemoryStore: Block broadcast_109 stored as values in memory
(estimated size 349.9 KiB, free 20.2 GiB)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 4.0 in stage 458.0 (TID 1874)
in 11848 ms on 10.183.232.45 (executor 2) (1/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 10.0 in stage 457.0 (TID 1862)
in 11856 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 9.0 in stage 457.0 (TID 1861)
in 11858 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:22:06 INFO MemoryStore: Block broadcast_109_piece0 stored as bytes in
memory (estimated size 126.0 KiB, free 20.2 GiB)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 8.0 in stage 457.0 (TID 1860)
in 11858 ms on 10.183.229.123 (executor 0) (11/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 3.0 in stage 458.0 (TID 1873)
in 11849 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 2.0 in stage 458.0 (TID 1872)
in 11850 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 5.0 in stage 458.0 (TID 1875)
in 11850 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 7.0 in stage 458.0 (TID 1877)
in 11850 ms on 10.183.249.197 (executor 7) (5/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 8.0 in stage 458.0 (TID 1878)
in 11850 ms on 10.183.249.197 (executor 7) (6/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 9.0 in stage 458.0 (TID 1879)
in 11850 ms on 10.183.249.197 (executor 7) (7/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 6.0 in stage 458.0 (TID 1876)
in 11850 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 17.0 in stage 457.0 (TID 1869)
in 11853 ms on 10.183.233.191 (executor 5) (12/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 12.0 in stage 457.0 (TID 1864)
in 11859 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 13.0 in stage 457.0 (TID 1865)
in 11860 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 15.0 in stage 457.0 (TID 1867)
in 11860 ms on 10.183.246.72 (executor 6) (15/18)
25/02/06 06:22:06 INFO TaskSetManager: Starting task 9.0 in stage 460.0 (TID 1897)
(10.183.246.72, executor 6, partition 9, PROCESS_LOCAL,
25/02/06 06:22:06 INFO TaskSetManager: Finished task 11.0 in stage 457.0 (TID 1863)
in 11863 ms on 10.183.229.123 (executor 0) (16/18)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 14.0 in stage 457.0 (TID 1866)
in 11862 ms on 10.183.246.72 (executor 6) (17/18)
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO TaskSetManager: Finished task 16.0 in stage 457.0 (TID 1868)
in 11859 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:22:06 INFO TaskSchedulerImpl: Removed TaskSet 457.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:06 INFO DAGScheduler: ShuffleMapStage 457
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
176.148 s
25/02/06 06:22:06 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:06 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 458, ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562,
ShuffleMapStage 577, ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550,
ShuffleMapStage 529, ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523,
ShuffleMapStage 502, ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475,
ShuffleMapStage 467, ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571,
ShuffleMapStage 463, ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465,
ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532,
ShuffleMapStage 511, ShuffleMapStage 613, ShuffleMapStage 461, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:06 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:06 INFO DAGScheduler: failed: Set()
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.232.44:37705 (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:22:06 INFO SparkContext: Created broadcast 109 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.249.197:40023 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.232.45:45579 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Removed broadcast_106_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.233.191:42923 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.246.72:43231 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:06 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(235,1738822301714,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@18a9be1c, org.apache.spark.scheduler.StageInfo@3b420bb6,
org.apache.spark.scheduler.StageInfo@55e34f5d,
org.apache.spark.scheduler.StageInfo@4b14ae78),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 12.089945489s.
25/02/06 06:22:16 INFO TaskSetManager: Finished task 0.0 in stage 458.0 (TID 1870)
in 22065 ms on 10.183.233.191 (executor 5) (9/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 1.0 in stage 458.0 (TID 1871)
in 22066 ms on 10.183.233.191 (executor 5) (10/18)
25/02/06 06:22:16 INFO TaskSetManager: Starting task 10.0 in stage 460.0 (TID 1898)
(10.183.233.191, executor 5, partition 10, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 11.0 in stage 460.0 (TID 1899)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Finished task 2.0 in stage 460.0 (TID 1890)
in 10250 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 15.0 in stage 458.0 (TID 1885)
in 10253 ms on 10.183.229.123 (executor 0) (11/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 5.0 in stage 460.0 (TID 1893)
in 10250 ms on 10.183.233.191 (executor 5) (2/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 16.0 in stage 458.0 (TID 1886)
in 10254 ms on 10.183.229.123 (executor 0) (12/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 14.0 in stage 458.0 (TID 1884)
in 10254 ms on 10.183.229.123 (executor 0) (13/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 11.0 in stage 458.0 (TID 1881)
in 10256 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:22:16 INFO TaskSetManager: Starting task 12.0 in stage 460.0 (TID 1900)
(10.183.233.191, executor 5, partition 12, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 13.0 in stage 460.0 (TID 1901)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 14.0 in stage 460.0 (TID 1902)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 15.0 in stage 460.0 (TID 1903)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 16.0 in stage 460.0 (TID 1904)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 17.0 in stage 460.0 (TID 1905)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 0.0 in stage 459.0 (TID 1906)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 1.0 in stage 459.0 (TID 1907)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 2.0 in stage 459.0 (TID 1908)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 3.0 in stage 459.0 (TID 1909)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:22:16 INFO MemoryStore: Block broadcast_110 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:22:16 INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in
memory (estimated size 126.0 KiB, free 20.2 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on
10.183.232.44:37705 (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:22:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:16 INFO SparkContext: Created broadcast 110 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:16 INFO TaskSetManager: Starting task 4.0 in stage 459.0 (TID 1910)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 5.0 in stage 459.0 (TID 1911)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 6.0 in stage 459.0 (TID 1912)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 7.0 in stage 459.0 (TID 1913)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Finished task 10.0 in stage 458.0 (TID 1880)
in 10277 ms on 10.183.246.72 (executor 6) (15/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 17.0 in stage 458.0 (TID 1887)
in 10275 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 9.0 in stage 460.0 (TID 1897)
in 10238 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 0.0 in stage 460.0 (TID 1888)
in 10275 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 3.0 in stage 460.0 (TID 1891)
in 10275 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 12.0 in stage 458.0 (TID 1882)
in 10278 ms on 10.183.246.72 (executor 6) (17/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 13.0 in stage 458.0 (TID 1883)
in 10277 ms on 10.183.229.123 (executor 0) (18/18)
25/02/06 06:22:16 INFO TaskSchedulerImpl: Removed TaskSet 458.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:16 INFO TaskSetManager: Finished task 6.0 in stage 460.0 (TID 1894)
in 10274 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 1.0 in stage 460.0 (TID 1889)
in 10276 ms on 10.183.249.197 (executor 7) (7/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 8.0 in stage 460.0 (TID 1896)
in 10275 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 7.0 in stage 460.0 (TID 1895)
in 10276 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:22:16 INFO TaskSetManager: Finished task 4.0 in stage 460.0 (TID 1892)
in 10277 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:22:16 INFO TaskSetManager: Starting task 8.0 in stage 459.0 (TID 1914)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:22:16 INFO DAGScheduler: ShuffleMapStage 458
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
186.363 s
25/02/06 06:22:16 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:16 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 460, ShuffleMapStage 562, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511,
ShuffleMapStage 613, ShuffleMapStage 461, ShuffleMapStage 505, ShuffleMapStage 484,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 574,
ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526,
ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:16 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:16 INFO DAGScheduler: failed: Set()
25/02/06 06:22:16 INFO TaskSetManager: Starting task 9.0 in stage 459.0 (TID 1915)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 10.0 in stage 459.0 (TID 1916)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:22:16 INFO TaskSetManager: Starting task 11.0 in stage 459.0 (TID 1917)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:22:16 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on
10.183.246.72:43231 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on
10.183.229.123:38441 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on
10.183.232.45:45579 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on
10.183.249.197:40023 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Removed broadcast_107_piece0 on
10.183.229.123:38441 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Removed broadcast_107_piece0 on
10.183.232.45:45579 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Removed broadcast_107_piece0 on
10.183.246.72:43231 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Removed broadcast_107_piece0 on
10.183.232.44:37705 in memory (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:22:16 INFO BlockManagerInfo: Removed broadcast_107_piece0 on
10.183.233.191:42923 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:16 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:16 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:16 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:16 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:16 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2100487) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2100487)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:22:16 INFO AsyncEventQueue: Process of event
SparkListenerSQLAdaptiveExecutionUpdate(executionId=32, ...) by listener
SQLAppStatusListener took 9.932326334s.
25/02/06 06:22:16 INFO CodeGenerator: Code generated in 132.65673 ms
25/02/06 06:22:16 INFO CodeGenerator: Code generated in 120.999476 ms
25/02/06 06:22:16 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683765)
25/02/06 06:22:16 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683045)
25/02/06 06:22:16 INFO DAGScheduler: Registering RDD 2002
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 306
25/02/06 06:22:16 INFO DAGScheduler: Got map stage job 309
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:22:16 INFO DAGScheduler: Final stage: ShuffleMapStage 624
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:22:16 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
622, ShuffleMapStage 623, ShuffleMapStage 620, ShuffleMapStage 621)
25/02/06 06:22:16 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 684233)
25/02/06 06:22:16 INFO DAGScheduler: Missing parents: List()
25/02/06 06:22:16 INFO DAGScheduler: Submitting ShuffleMapStage 624
(MapPartitionsRDD[2002] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:22:16 INFO DAGScheduler: Registering RDD 2004
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 307
25/02/06 06:22:16 INFO DAGScheduler: Got map stage job 310
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:22:16 INFO DAGScheduler: Final stage: ShuffleMapStage 635
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:22:16 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
633, ShuffleMapStage 634, ShuffleMapStage 631, ShuffleMapStage 632)
25/02/06 06:22:16 INFO DAGScheduler: Missing parents: List()
25/02/06 06:22:16 INFO DAGScheduler: Submitting ShuffleMapStage 635
(MapPartitionsRDD[2004] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:22:16 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 624 (MapPartitionsRDD[2002] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:22:16 INFO TaskSchedulerImpl: Adding task set 624.0 with 1 tasks
resource profile 0
25/02/06 06:22:16 INFO TaskSetManager: TaskSet 624.0 using PreferredLocationsV1
25/02/06 06:22:16 INFO FairSchedulableBuilder: Added task set TaskSet_624.0 tasks
to pool 5400065285585087590
25/02/06 06:22:16 WARN BlockManager: Asked to remove block join_relation_683765,
which does not exist
25/02/06 06:22:16 WARN BlockManager: Asked to remove block join_relation_683045,
which does not exist
25/02/06 06:22:16 WARN BlockManager: Asked to remove block join_relation_684233,
which does not exist
25/02/06 06:22:16 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 635 (MapPartitionsRDD[2004] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:22:16 INFO TaskSchedulerImpl: Adding task set 635.0 with 1 tasks
resource profile 0
25/02/06 06:22:16 INFO TaskSetManager: TaskSet 635.0 using PreferredLocationsV1
25/02/06 06:22:16 INFO FairSchedulableBuilder: Added task set TaskSet_635.0 tasks
to pool 5400065285585087590
25/02/06 06:22:24 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:22:24 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:22:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:24 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:24 INFO TaskSetManager: Starting task 12.0 in stage 459.0 (TID 1918)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Finished task 1.0 in stage 459.0 (TID 1907)
in 8332 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:22:24 INFO TaskSetManager: Starting task 13.0 in stage 459.0 (TID 1919)
(10.183.232.45, executor 2, partition 13, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 14.0 in stage 459.0 (TID 1920)
(10.183.232.45, executor 2, partition 14, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 15.0 in stage 459.0 (TID 1921)
(10.183.232.45, executor 2, partition 15, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 16.0 in stage 459.0 (TID 1922)
(10.183.232.45, executor 2, partition 16, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 17.0 in stage 459.0 (TID 1923)
(10.183.246.72, executor 6, partition 17, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 0.0 in stage 461.0 (TID 1924)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 1.0 in stage 461.0 (TID 1925)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Finished task 5.0 in stage 459.0 (TID 1911)
in 8331 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:22:24 INFO MemoryStore: Block broadcast_111 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 3.0 in stage 459.0 (TID 1909)
in 8340 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 4.0 in stage 459.0 (TID 1910)
in 8333 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:22:24 INFO MemoryStore: Block broadcast_111_piece0 stored as bytes in
memory (estimated size 126.3 KiB, free 20.2 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on
10.183.232.44:37705 (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 2.0 in stage 459.0 (TID 1908)
in 8346 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO SparkContext: Created broadcast 111 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:24 INFO TaskSetManager: Finished task 17.0 in stage 460.0 (TID 1905)
in 8350 ms on 10.183.229.123 (executor 0) (11/18)
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 15.0 in stage 460.0 (TID 1903)
in 8351 ms on 10.183.229.123 (executor 0) (12/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 16.0 in stage 460.0 (TID 1904)
in 8352 ms on 10.183.229.123 (executor 0) (13/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 14.0 in stage 460.0 (TID 1902)
in 8352 ms on 10.183.229.123 (executor 0) (14/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 7.0 in stage 459.0 (TID 1913)
in 8342 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 6.0 in stage 459.0 (TID 1912)
in 8344 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 10.0 in stage 459.0 (TID 1916)
in 8330 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 9.0 in stage 459.0 (TID 1915)
in 8330 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 8.0 in stage 459.0 (TID 1914)
in 8331 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 0.0 in stage 459.0 (TID 1906)
in 8355 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 11.0 in stage 459.0 (TID 1917)
in 8330 ms on 10.183.249.197 (executor 7) (12/18)
25/02/06 06:22:24 INFO TaskSetManager: Starting task 2.0 in stage 461.0 (TID 1926)
(10.183.229.123, executor 0, partition 2, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 3.0 in stage 461.0 (TID 1927)
(10.183.229.123, executor 0, partition 3, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 4.0 in stage 461.0 (TID 1928)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 5.0 in stage 461.0 (TID 1929)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 6.0 in stage 461.0 (TID 1930)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 7.0 in stage 461.0 (TID 1931)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 8.0 in stage 461.0 (TID 1932)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 9.0 in stage 461.0 (TID 1933)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.232.45:45579 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:22:24 INFO TaskSetManager: Starting task 10.0 in stage 461.0 (TID 1934)
(10.183.233.191, executor 5, partition 10, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 11.0 in stage 461.0 (TID 1935)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 12.0 in stage 461.0 (TID 1936)
(10.183.233.191, executor 5, partition 12, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Starting task 13.0 in stage 461.0 (TID 1937)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:22:24 INFO TaskSetManager: Finished task 13.0 in stage 460.0 (TID 1901)
in 8407 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 12.0 in stage 460.0 (TID 1900)
in 8409 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 10.0 in stage 460.0 (TID 1898)
in 8416 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:22:24 INFO TaskSetManager: Finished task 11.0 in stage 460.0 (TID 1899)
in 8416 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:22:24 INFO TaskSchedulerImpl: Removed TaskSet 460.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:24 INFO DAGScheduler: ShuffleMapStage 460
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
194.734 s
25/02/06 06:22:24 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:24 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 459, ShuffleMapStage 532, ShuffleMapStage 511,
ShuffleMapStage 613, ShuffleMapStage 461, ShuffleMapStage 505, ShuffleMapStage 484,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 624,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:24 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:24 INFO DAGScheduler: failed: Set()
25/02/06 06:22:24 INFO BlockManagerInfo: Removed broadcast_108_piece0 on
10.183.233.191:42923 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on
10.183.229.123:38441 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on
10.183.249.197:40023 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on
10.183.246.72:43231 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:24 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on
10.183.233.191:42923 (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:25 INFO ShufflePartitionsUtil: For shuffle(254), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:22:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:25 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(236,1738822477072,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@4f6fea5b, org.apache.spark.scheduler.StageInfo@1ae3f4f4,
org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@315753a5),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 8.55415052s.
25/02/06 06:22:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:35 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:35 INFO TaskSetManager: Finished task 2.0 in stage 461.0 (TID 1926)
in 10619 ms on 10.183.229.123 (executor 0) (1/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 4.0 in stage 461.0 (TID 1928)
in 10620 ms on 10.183.229.123 (executor 0) (2/18)
25/02/06 06:22:35 INFO TaskSetManager: Starting task 14.0 in stage 461.0 (TID 1938)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 15.0 in stage 461.0 (TID 1939)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Finished task 5.0 in stage 461.0 (TID 1929)
in 10623 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 3.0 in stage 461.0 (TID 1927)
in 10624 ms on 10.183.229.123 (executor 0) (4/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 10.0 in stage 461.0 (TID 1934)
in 10579 ms on 10.183.233.191 (executor 5) (5/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 12.0 in stage 461.0 (TID 1936)
in 10578 ms on 10.183.233.191 (executor 5) (6/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 13.0 in stage 461.0 (TID 1937)
in 10579 ms on 10.183.233.191 (executor 5) (7/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 16.0 in stage 459.0 (TID 1922)
in 10651 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 15.0 in stage 459.0 (TID 1921)
in 10652 ms on 10.183.232.45 (executor 2) (14/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 14.0 in stage 459.0 (TID 1920)
in 10652 ms on 10.183.232.45 (executor 2) (15/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 13.0 in stage 459.0 (TID 1919)
in 10653 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:22:35 INFO TaskSetManager: Starting task 16.0 in stage 461.0 (TID 1940)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 17.0 in stage 461.0 (TID 1941)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 0.0 in stage 462.0 (TID 1942)
(10.183.233.191, executor 5, partition 0, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 1.0 in stage 462.0 (TID 1943)
(10.183.233.191, executor 5, partition 1, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 2.0 in stage 462.0 (TID 1944)
(10.183.233.191, executor 5, partition 2, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 3.0 in stage 462.0 (TID 1945)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 4.0 in stage 462.0 (TID 1946)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 5.0 in stage 462.0 (TID 1947)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 6.0 in stage 462.0 (TID 1948)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 7.0 in stage 462.0 (TID 1949)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:22:35 INFO MemoryStore: Block broadcast_112 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.233.191:42923 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.232.45:45579 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.249.197:40023 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.246.72:43231 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.229.123:38441 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO TaskSetManager: Starting task 8.0 in stage 462.0 (TID 1950)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 9.0 in stage 462.0 (TID 1951)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 10.0 in stage 462.0 (TID 1952)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 11.0 in stage 462.0 (TID 1953)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Finished task 6.0 in stage 461.0 (TID 1930)
in 10656 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 8.0 in stage 461.0 (TID 1932)
in 10656 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 9.0 in stage 461.0 (TID 1933)
in 10656 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 17.0 in stage 459.0 (TID 1923)
in 10680 ms on 10.183.246.72 (executor 6) (17/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 12.0 in stage 459.0 (TID 1918)
in 10685 ms on 10.183.246.72 (executor 6) (18/18)
25/02/06 06:22:35 INFO TaskSchedulerImpl: Removed TaskSet 459.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:35 INFO DAGScheduler: ShuffleMapStage 459
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
205.346 s
25/02/06 06:22:35 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:35 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 613,
ShuffleMapStage 461, ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 624, ShuffleMapStage 574,
ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526,
ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 464, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:35 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:35 INFO DAGScheduler: failed: Set()
25/02/06 06:22:35 INFO TaskSetManager: Finished task 7.0 in stage 461.0 (TID 1931)
in 10665 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:22:35 INFO TaskSetManager: Finished task 11.0 in stage 461.0 (TID 1935)
in 10619 ms on 10.183.233.191 (executor 5) (12/18)
25/02/06 06:22:35 INFO TaskSetManager: Starting task 12.0 in stage 462.0 (TID 1954)
(10.183.246.72, executor 6, partition 12, PROCESS_LOCAL,
25/02/06 06:22:35 INFO TaskSetManager: Starting task 13.0 in stage 462.0 (TID 1955)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:22:35 INFO MemoryStore: Block broadcast_112_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Removed broadcast_109_piece0 on
10.183.232.44:37705 in memory (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:22:35 INFO SparkContext: Created broadcast 112 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:35 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:22:35 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on
10.183.246.72:43231 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 1.0 in stage 462.0 (TID 1943)
in 9588 ms on 10.183.233.191 (executor 5) (1/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 6.0 in stage 462.0 (TID 1948)
in 9588 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 1.0 in stage 461.0 (TID 1925)
in 20255 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 7.0 in stage 462.0 (TID 1949)
in 9588 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 3.0 in stage 462.0 (TID 1945)
in 9590 ms on 10.183.233.191 (executor 5) (4/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 13.0 in stage 462.0 (TID 1955)
in 9567 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:22:44 INFO TaskSetManager: Starting task 14.0 in stage 462.0 (TID 1956)
(10.183.233.191, executor 5, partition 14, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 15.0 in stage 462.0 (TID 1957)
(10.183.233.191, executor 5, partition 15, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 16.0 in stage 462.0 (TID 1958)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 17.0 in stage 462.0 (TID 1959)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 0.0 in stage 463.0 (TID 1960)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 1.0 in stage 463.0 (TID 1961)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 2.0 in stage 463.0 (TID 1962)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 3.0 in stage 463.0 (TID 1963)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 4.0 in stage 463.0 (TID 1964)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 5.0 in stage 463.0 (TID 1965)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 6.0 in stage 463.0 (TID 1966)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 7.0 in stage 463.0 (TID 1967)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:22:44 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:44 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:44 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:44 INFO MemoryStore: Block broadcast_113 stored as values in memory
(estimated size 349.7 KiB, free 20.2 GiB)
25/02/06 06:22:44 INFO TaskSetManager: Starting task 8.0 in stage 463.0 (TID 1968)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 9.0 in stage 463.0 (TID 1969)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 10.0 in stage 463.0 (TID 1970)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Starting task 11.0 in stage 463.0 (TID 1971)
(10.183.249.197, executor 7, partition 11, PROCESS_LOCAL,
25/02/06 06:22:44 INFO TaskSetManager: Finished task 2.0 in stage 462.0 (TID 1944)
in 9595 ms on 10.183.233.191 (executor 5) (6/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 0.0 in stage 462.0 (TID 1942)
in 9596 ms on 10.183.233.191 (executor 5) (7/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 5.0 in stage 462.0 (TID 1947)
in 9595 ms on 10.183.232.45 (executor 2) (8/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 8.0 in stage 462.0 (TID 1950)
in 9585 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 11.0 in stage 462.0 (TID 1953)
in 9585 ms on 10.183.249.197 (executor 7) (10/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 9.0 in stage 462.0 (TID 1951)
in 9586 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 0.0 in stage 461.0 (TID 1924)
in 20265 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:22:44 INFO BlockManagerInfo: Removed broadcast_110_piece0 on
10.183.246.72:43231 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:44 INFO TaskSetManager: Finished task 4.0 in stage 462.0 (TID 1946)
in 9601 ms on 10.183.232.45 (executor 2) (12/18)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 10.0 in stage 462.0 (TID 1952)
in 9592 ms on 10.183.249.197 (executor 7) (13/18)
25/02/06 06:22:45 INFO BlockManagerInfo: Removed broadcast_110_piece0 on
10.183.232.45:45579 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Removed broadcast_110_piece0 on
10.183.249.197:40023 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 12.0 in stage 462.0 (TID 1954)
in 9588 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 17.0 in stage 461.0 (TID 1941)
in 9612 ms on 10.183.229.123 (executor 0) (15/18)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 14.0 in stage 461.0 (TID 1938)
in 9636 ms on 10.183.229.123 (executor 0) (16/18)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 16.0 in stage 461.0 (TID 1940)
in 9613 ms on 10.183.229.123 (executor 0) (17/18)
25/02/06 06:22:45 INFO TaskSetManager: Finished task 15.0 in stage 461.0 (TID 1939)
in 9636 ms on 10.183.229.123 (executor 0) (18/18)
25/02/06 06:22:45 INFO TaskSchedulerImpl: Removed TaskSet 461.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:45 INFO DAGScheduler: ShuffleMapStage 461
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
214.910 s
25/02/06 06:22:45 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:45 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 613,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 580, ShuffleMapStage 624, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 462, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:45 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:45 INFO DAGScheduler: failed: Set()
25/02/06 06:22:45 INFO TaskSetManager: Starting task 12.0 in stage 463.0 (TID 1972)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:22:45 INFO TaskSetManager: Starting task 13.0 in stage 463.0 (TID 1973)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:22:45 INFO TaskSetManager: Starting task 14.0 in stage 463.0 (TID 1974)
(10.183.229.123, executor 0, partition 14, PROCESS_LOCAL,
25/02/06 06:22:45 INFO TaskSetManager: Starting task 15.0 in stage 463.0 (TID 1975)
(10.183.229.123, executor 0, partition 15, PROCESS_LOCAL,
25/02/06 06:22:45 INFO MemoryStore: Block broadcast_113_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:22:45 INFO SparkContext: Created broadcast 113 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:45 INFO BlockManagerInfo: Removed broadcast_110_piece0 on
10.183.232.44:37705 in memory (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:22:45 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(237,1738822509435,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@7e817b75, org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@4e0c6bcc,
org.apache.spark.scheduler.StageInfo@4b4d72ad),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 19.76375359s.
25/02/06 06:22:45 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 685989)
25/02/06 06:22:45 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:45 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:22:45 WARN BlockManager: Asked to remove block join_relation_685989,
which does not exist
25/02/06 06:22:45 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683181)
25/02/06 06:22:45 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2099417) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2099417)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:22:45 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682677)
25/02/06 06:22:45 WARN BlockManager: Asked to remove block join_relation_683181,
which does not exist
25/02/06 06:22:45 WARN BlockManager: Asked to remove block join_relation_682677,
which does not exist
25/02/06 06:22:45 INFO CodeGenerator: Code generated in 115.380861 ms
25/02/06 06:22:45 INFO DAGScheduler: Registering RDD 2022
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) as input to
shuffle 308
25/02/06 06:22:45 INFO DAGScheduler: Got map stage job 311
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) with 1 output
partitions
25/02/06 06:22:45 INFO DAGScheduler: Final stage: ShuffleMapStage 646
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63)
25/02/06 06:22:45 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage
643, ShuffleMapStage 644, ShuffleMapStage 645, ShuffleMapStage 642)
25/02/06 06:22:45 INFO DAGScheduler: Missing parents: List()
25/02/06 06:22:45 INFO DAGScheduler: Submitting ShuffleMapStage 646
(MapPartitionsRDD[2022] at $anonfun$withThreadLocalCaptured$5 at
LexicalThreadLocal.scala:63), which has no missing parents
25/02/06 06:22:45 INFO DAGScheduler: Submitting 1 missing tasks from
ShuffleMapStage 646 (MapPartitionsRDD[2022] at $anonfun$withThreadLocalCaptured$5
at LexicalThreadLocal.scala:63) (first 15 tasks are for partitions Vector(0))
25/02/06 06:22:45 INFO TaskSchedulerImpl: Adding task set 646.0 with 1 tasks
resource profile 0
25/02/06 06:22:45 INFO TaskSetManager: TaskSet 646.0 using PreferredLocationsV1
25/02/06 06:22:45 INFO FairSchedulableBuilder: Added task set TaskSet_646.0 tasks
to pool 5400065285585087590
25/02/06 06:22:45 INFO ShufflePartitionsUtil: For shuffle(256), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:22:45 INFO ShufflePartitionsUtil: For shuffle(252), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:22:45 INFO ShufflePartitionsUtil: For shuffle(248), advisory target
size: 67108864, actual target size 1048576, minimum partition size: 1048576
25/02/06 06:22:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:55 INFO BlockManagerInfo: Removed broadcast_111_piece0 on
10.183.233.191:42923 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO BlockManagerInfo: Removed broadcast_111_piece0 on
10.183.246.72:43231 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO BlockManagerInfo: Removed broadcast_111_piece0 on
10.183.229.123:38441 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO TaskSetManager: Starting task 16.0 in stage 463.0 (TID 1976)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 17.0 in stage 463.0 (TID 1977)
(10.183.229.123, executor 0, partition 17, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 0.0 in stage 464.0 (TID 1978)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 1.0 in stage 464.0 (TID 1979)
(10.183.229.123, executor 0, partition 1, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 2.0 in stage 464.0 (TID 1980)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 3.0 in stage 464.0 (TID 1981)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 4.0 in stage 464.0 (TID 1982)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 5.0 in stage 464.0 (TID 1983)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 6.0 in stage 464.0 (TID 1984)
(10.183.233.191, executor 5, partition 6, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 7.0 in stage 464.0 (TID 1985)
(10.183.233.191, executor 5, partition 7, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 8.0 in stage 464.0 (TID 1986)
(10.183.233.191, executor 5, partition 8, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 9.0 in stage 464.0 (TID 1987)
(10.183.233.191, executor 5, partition 9, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Finished task 12.0 in stage 463.0 (TID 1972)
in 10902 ms on 10.183.229.123 (executor 0) (1/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 6.0 in stage 463.0 (TID 1966)
in 10926 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 5.0 in stage 463.0 (TID 1965)
in 10936 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 17.0 in stage 462.0 (TID 1959)
in 10938 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 15.0 in stage 462.0 (TID 1957)
in 10939 ms on 10.183.233.191 (executor 5) (16/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 16.0 in stage 462.0 (TID 1958)
in 10939 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 14.0 in stage 462.0 (TID 1956)
in 10943 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:22:55 INFO TaskSchedulerImpl: Removed TaskSet 462.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:22:55 INFO TaskSetManager: Finished task 4.0 in stage 463.0 (TID 1964)
in 10943 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 2.0 in stage 463.0 (TID 1962)
in 10944 ms on 10.183.246.72 (executor 6) (5/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 3.0 in stage 463.0 (TID 1963)
in 10944 ms on 10.183.246.72 (executor 6) (6/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 7.0 in stage 463.0 (TID 1967)
in 10943 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 0.0 in stage 463.0 (TID 1960)
in 10944 ms on 10.183.246.72 (executor 6) (8/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 13.0 in stage 463.0 (TID 1973)
in 10921 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 14.0 in stage 463.0 (TID 1974)
in 10921 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 15.0 in stage 463.0 (TID 1975)
in 10921 ms on 10.183.229.123 (executor 0) (11/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 9.0 in stage 463.0 (TID 1969)
in 10943 ms on 10.183.249.197 (executor 7) (12/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 11.0 in stage 463.0 (TID 1971)
in 10943 ms on 10.183.249.197 (executor 7) (13/18)
25/02/06 06:22:55 INFO MemoryStore: Block broadcast_114 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:22:55 INFO DAGScheduler: ShuffleMapStage 462
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
225.818 s
25/02/06 06:22:55 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:22:55 INFO BlockManagerInfo: Removed broadcast_111_piece0 on
10.183.232.44:37705 in memory (size: 126.3 KiB, free: 20.2 GiB)
25/02/06 06:22:55 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 463,
ShuffleMapStage 565, ShuffleMapStage 544, ShuffleMapStage 646, ShuffleMapStage 465,
ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 532, ShuffleMapStage 511,
ShuffleMapStage 613, ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499,
ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 624, ShuffleMapStage 574,
ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526,
ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520,
ShuffleMapStage 470, ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:22:55 INFO DAGScheduler: waiting: Set()
25/02/06 06:22:55 INFO DAGScheduler: failed: Set()
25/02/06 06:22:55 INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in
memory (estimated size 126.4 KiB, free 20.2 GiB)
25/02/06 06:22:55 INFO TaskSetManager: Starting task 10.0 in stage 464.0 (TID 1988)
(10.183.249.197, executor 7, partition 10, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 11.0 in stage 464.0 (TID 1989)
(10.183.246.72, executor 6, partition 11, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 12.0 in stage 464.0 (TID 1990)
(10.183.249.197, executor 7, partition 12, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 13.0 in stage 464.0 (TID 1991)
(10.183.246.72, executor 6, partition 13, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 14.0 in stage 464.0 (TID 1992)
(10.183.249.197, executor 7, partition 14, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 15.0 in stage 464.0 (TID 1993)
(10.183.246.72, executor 6, partition 15, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 16.0 in stage 464.0 (TID 1994)
(10.183.249.197, executor 7, partition 16, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Starting task 17.0 in stage 464.0 (TID 1995)
(10.183.246.72, executor 6, partition 17, PROCESS_LOCAL,
25/02/06 06:22:55 INFO TaskSetManager: Finished task 10.0 in stage 463.0 (TID 1970)
in 10957 ms on 10.183.249.197 (executor 7) (14/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 1.0 in stage 463.0 (TID 1961)
in 10959 ms on 10.183.246.72 (executor 6) (15/18)
25/02/06 06:22:55 INFO TaskSetManager: Finished task 8.0 in stage 463.0 (TID 1968)
in 10957 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:22:55 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.232.44:37705 (size: 126.4 KiB, free: 20.2 GiB)
25/02/06 06:22:55 INFO BlockManagerInfo: Removed broadcast_111_piece0 on
10.183.249.197:40023 in memory (size: 126.3 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO SparkContext: Created broadcast 114 from broadcast at
TaskSetManager.scala:711
25/02/06 06:22:55 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.229.123:38441 (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.246.72:43231 (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:22:55 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.233.191:42923 (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:22:56 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.232.45:45579 (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:22:56 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on
10.183.249.197:40023 (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:22:56 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:22:56 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:22:56 INFO ProgressReporter$: Reporting partial results for running
commands: 5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63ab466b9
[47 occurrences]
25/02/06 06:22:56 INFO AsyncEventQueue: Process of event
SparkListenerJobStart(238,1738822567107,WrappedArray(org.apache.spark.scheduler.Sta
geInfo@408cca1a, org.apache.spark.scheduler.StageInfo@4b14ae78,
org.apache.spark.scheduler.StageInfo@35234bad,
org.apache.spark.scheduler.StageInfo@185c04ce),
{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem,
spark.databricks.dbfsFusePrivaceraIntegration=false,
spark.databricks.clusterUsageTags.clusterEbsVolumeCount=1,
spark.databricks.session.share=false,
spark.hadoop.hive.server2.session.check.interval=60000,
spark.databricks.clusterUsageTags.clusterPolicyId=001FFD181EB34513,
spark.databricks.pyspark.pythonUdfsOnly=true,
spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore
,
spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.Manag
edCatalogClientImpl, spark.driver.port=36145, spark.executor.extraJavaOptions=-
Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-
opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-
UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false -Djava.io.tmpdir=/local_disk0/tmp -
XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -
XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -
Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-
UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails
-verbose:gc -Xss4m
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/
usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/
lib/jni -
Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatyp
e.DatatypeFactoryImpl -
Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.D
ocumentBuilderFactoryImpl -
Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXPars
erFactoryImpl
-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.a
pache.xerces.internal.jaxp.validation.XMLSchemaFactory -
Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -
Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOM
XSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -
Dscala.reflect.runtime.disable.typetag.cache=true -
Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=tru
e -Dio.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -
Ddatabricks.serviceName=spark-executor-1,
spark.databricks.clusterUsageTags.sparkMasterUrlType=None,
spark.databricks.acl.checkPermission.impl=com.databricks.sql.acl.CheckPermissions,
spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false,
spark.hadoop.fs.cpfs-
s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.mlflow.autologging.enabled=true,
spark.databricks.clusterUsageTags.clusterCreator=Webapp,
spark.databricks.clusterUsageTags.enableSqlAclsOnly=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=0,
spark.sql.sources.default=delta,
spark.databricks.daemon.driver.python.enableDevbricks=false,
spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStor
e, spark.driver.extraJavaOptions=-Djava.net.preferIPv6Addresses=false -XX:
+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-
opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-
UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-
opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-
opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-
opens=java.management/sun.management=ALL-UNNAMED -
Djdk.reflect.useDirectMethodHandle=false, spark.task.reaper.killTimeout=60s,
spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
__fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true,
spark.hadoop.fs.s3a.max.total.tasks=1000,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=ebs_volume_type:
GENERAL_PURPOSE_SSD
, spark.home=/databricks/spark,
spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.metrics.filesystem_io_metrics=true,
spark.hadoop.fs.stage.impl.disable.cache=true,
spark.databricks.unityCatalog.userIsolation.python.enabled=false,
spark.databricks.clusterUsageTags.clusterAvailability=SPOT_WITH_FALLBACK,
spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.
DirectParquetOutputCommitter,
spark.databricks.io.directoryCommit.enableLogicalDelete=false,
spark.app.startTime=1738821359344, spark.executor.tempDirectory=/local_disk0/tmp,
spark.hadoop.fs.s3a.connection.maximum=200,
spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databrick
s.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider,
spark.databricks.clusterUsageTags.clusterLogDestinationType=,
spark.databricks.clusterUsageTags.driverInstanceId=i-00f4ec534ef0e57a4,
spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceau
tocompute.MACClientImpl, spark.databricks.acl.hiveNamespaceOnly=true,
spark.sql.optimizer.optimizeCsvJsonExprs.useSchemaField=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_bazel_path=,
spark.databricks.clusterUsageTags.userProvidedSparkVersion=13.3.x-scala2.12,
spark.hadoop.fs.gs.impl.disable.cache=true,
spark.databricks.sparkContextId=9206972423021441211,
spark.databricks.clusterUsageTags.clusterSpotBidPricePercent=100,
spark.databricks.workerNodeTypeId=r6i.xlarge,
spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false,
spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2,
spark.databricks.clusterUsageTags.clusterId=1019-090756-ocity0wb,
spark.databricks.privateLinkEnabled=false,
spark.databricks.clusterUsageTags.runtimeEngine=PHOTON,
spark.job.description=df_online.show(),
spark.databricks.clusterUsageTags.clusterWorkers=5, spark.app.id=app-
20250206055605-0000,
spark.databricks.clusterUsageTags.enableCredentialPassthrough=false,
spark.databricks.clusterUsageTags.clusterLastActivityTime=1738816411486,
spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databrick
s.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory,
spark.extraListeners=com.databricks.backend.daemon.driver.DBCEventLoggingListener,
spark.databricks.cloudfetch.requestDownloadUrlsWithHeaders=true,
spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT,
spark.databricks.streaming.enforceAllowlistOnSharedCluster=true,
spark.hadoop.parquet.page.verify-checksum.enabled=true,
spark.connect.extensions.command.classes=io.delta.connect.DeltaCommandPlugin,
spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.183.232.44,
spark.driver.tempDirectory=/local_disk0/tmp,
spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-9206972423021441211-
35581f0d-a828-4acc-a93f-9fe7fbdc2046,
spark.hadoop.fs.wasbs.impl.disable.cache=true,
spark.databricks.pyspark.trustedFilesystems=com.databricks.adl.AdlFileSystem,com.da
tabricks.s3a.S3AFileSystem,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
,shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystemHadoop3,shaded.databricks.
org.apache.hadoop.fs.azure.NativeAzureFileSystem,shaded.databricks.v20180920_b33d81
0.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem,shaded.databricks.azurebf
s.org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystemHadoop3,shaded.databricks.
V2_1_4.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.
google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem,shaded.databricks.com.google.clou
d.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3,com.databricks.common.filesystem.Loki
FileSystem,com.databricks.common.filesystem.LokiS3FS,com.databricks.common.filesyst
em.LokiABFS,com.databricks.common.filesystem.LokiSecureABFS,com.databricks.common.f
ilesystem.LokiGCFS,org.apache.spark.sql.execution.datasources.jdbc.JdbcFileSystem,
spark.shuffle.service.enabled=true, spark.speculation=false,
spark.eventLog.enabled=false,
spark.databricks.clusterUsageTags.clusterHasPolicy=true,
spark.hadoop.spark.databricks.metrics.filesystem_metrics=true,
spark.databricks.delta.clone.es1281016.useFullDVPathForDiff=false,
spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-
s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-
s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_budget=,
spark.databricks.clusterUsageTags.userId=1687111647522361,
spark.databricks.telemetry.prometheus.samplingRate=100,
spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.d
ata.client.MapReduceDirectOutputCommitter,
spark.databricks.clusterUsageTags.clusterTargetWorkers=5,
spark.hadoop.fs.adl.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000,
spark.sql.hive.convertCTAS=true,
spark.databricks.hive.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.Hi
veCheckFileSystemSafety,
spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.dri
ver.credentials.OAuthTokenRefresherClient,
spark.databricks.clusterUsageTags.containerType=LXC,
spark.databricks.secret.envVar.keys.toRedact=,
spark.databricks.unityCatalog.enabled=true, spark.ui.prometheus.enabled=true,
spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true,
spark.databricks.unityCatalog.volumes.enabled=true,
spark.databricks.clusterUsageTags.isGroupCluster=false,
spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.Pers
onalStagingFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true,
spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem,
unity.scope.data=rO0ABXNyACRjb20uZGF0YWJyaWNrcy51bml0eS5VQ1NEcml2ZXIkU2NvcGXEH4f5HP
KlhQIABkwADERlYnVnJG1vZHVsZXQAMkxjb20vZGF0YWJyaWNrcy91bml0eS9Vbml0eUNyZWRlbnRpYWxTY
29wZSREZWJ1ZyQ7TAASYXR0cmlidXRpb25Db250ZXh0dAAuTGNvbS9kYXRhYnJpY2tzL3VuaXR5L1VuaXR5
QXR0cmlidXRpb25Db250ZXh0O0wAEWNyZWRlbnRpYWxNYW5hZ2VydAAoTGNvbS9kYXRhYnJpY2tzL3VuaXR
5L0NyZWRlbnRpYWxNYW5hZ2VyO0wAAmlkdAASTGphdmEvbGFuZy9TdHJpbmc7TAAQcGlwZWxpbmVSZWdpc3
RyeXQAJ0xjb20vZGF0YWJyaWNrcy91bml0eS9QaXBlbGluZVJlZ2lzdHJ5O0wAC3NhbVJlZ2lzdHJ5dAAiT
GNvbS9kYXRhYnJpY2tzL3VuaXR5L1NBTVJlZ2lzdHJ5O3hwcHNyADFjb20uZGF0YWJyaWNrcy51bml0eS5V
bml0eUF0dHJpYnV0aW9uQ29udGV4dENsYXNzj4Nk4oqFAxQCAAFMABVhdHRyaWJ1dGlvbkNvbnRleHRPcHR
0AC1MamF2YS91dGlsL2NvbmN1cnJlbnQvYXRvbWljL0F0b21pY1JlZmVyZW5jZTt4cHNyACtqYXZhLnV0aW
wuY29uY3VycmVudC5hdG9taWMuQXRvbWljUmVmZXJlbmNl5ldx1FV4VMYCAAFMAAV2YWx1ZXQAEkxqYXZhL
2xhbmcvT2JqZWN0O3hwc3IACnNjYWxhLlNvbWURIvJpXqGLdAIAAUwABXZhbHVlcQB+AAx4cgAMc2NhbGEu
T3B0aW9u/mk3/
dsOZnQCAAB4cHNyADVzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5IYXNoTWFwJFNlcmlhbGl6YXRpb25Q
cm94eQAAAAAAAAACAwAAeHB3BAAAAEh0AARvcElkdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2Z0AAlzaGF
yZE5hbWV0AAxpcmVsYW5kLXByb2R0AAhvcFRhcmdldHQAFGV4ZWN1dGVDb21tYW5kSW5SZXBsdAAWbm90ZW
Jvb2tFZGl0b3JOb2RlVHlwZXQABXNoZWxsdAANY2x1c3Rlck1lbW9yeXQABTE2Mzg0dAARc2VydmVyQmFja
2VuZE5hbWV0ADFjb20uZGF0YWJyaWNrcy5iYWNrZW5kLmRhZW1vbi5kcml2ZXIuRHJpdmVyQ29ycmFsdAAK
bm90ZWJvb2tJZHQADzU4OTE0NzgzODcxMDk2MXQAC3Byb2plY3ROYW1ldAAGZHJpdmVydAAEdGllcnQAEHR
pZXItbXVsdGl0ZW5hbnR0AB1zcWxXYXJlaG91c2VBc3NldFR5cGVPdmVycmlkZXQACG5vdGVib29rdAAPZX
ZlbnRXaW5kb3dUaW1ldAAJMjg5Nzk1Mi40dAAKaHR0cFRhcmdldHQAMi9ub3RlYm9vay81ODkxNDc4Mzg3M
TA5NjEvY29tbWFuZC82NDUzNjM3MTc4MzAxNzU3dAAMY29tbWFuZFJ1bklkdAAkOTMxNzkyY2MtMGI0ZS00
NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAJYnVpbGRIYXNodAAAdAAIdGVzdFVzZXJ0AA9jX2Rldi5jQGdtZy5
jb210ABZ3b3Jrc3BhY2VSb3V0aW5nVGFyZ2V0dAAEbnVsbHQAC2Jyb3dzZXJIYXNodAAZI2NvbW1hbmQvNj
Q1MzYzNzE3ODMwMTgwNHQAD2Jyb3dzZXJQYXRoTmFtZXQAIS9lZGl0b3Ivbm90ZWJvb2tzLzU4OTE0NzgzO
DcxMDk2MXQAEm5vdGVib29rRWRpdG9yTW9kZXQACG5vdGVib29rdAAQbm90ZWJvb2tMYW5ndWFnZXQABnB5
dGhvbnQAFXNxbFdhcmVob3VzZVF1ZXJ5VXVpZHEAfgAudAAWd29ya3NwYWNlUm91dGluZ0J1Y2tldHQABG5
1bGx0ABNwYXJlbnRTZXJ2ZXJFdmVudElkdAAsQ2d3SXBhT1J2UVlRc2JLbTFBSTZFT09CWVhqWmJrZU1pRE
pzbUFSblA4WT10AAxzcGFya1ZlcnNpb250ABAxMy4zLngtc2NhbGEyLjEydAASZXhlY3V0aW9uQ29udGV4d
ElkdAATNDU0MzEwOTk4OTkxODEwNTc5MHQACGhvc3ROYW1ldAANY29ucy13ZWJhcHAtM3QACmh0dHBNZXRo
b2R0AARQT1NUdAAPYnJvd3NlcklkbGVUaW1ldAASMzQyLjM5OTk5OTk5OTkwNjg3dAAUamV0dHlScGNKZXR
0eVZlcnNpb250AAE5dAAPYnJvd3Nlckxhbmd1YWdldAAFZW4tR0J0AAxicm93c2VyVGFiSWR0ACRlNThmM2
FlMS0xNzhhLTQ4YmQtOTY0NS03MTA5MWNhNmUyZDd0AA9zb3VyY2VJcEFkZHJlc3N0AA8xNTcuMTE5LjIxM
y4yNTF0AAlhY2NvdW50SWR0ACQxMTIxNTZiNy01NjkzLTQyNzQtOGViYy01ZWZjMGE5OThmNGZ0ABBsb2Fk
ZWRVaVZlcnNpb25zdAA5TWFwKG1vbm9saXRoIC0+IDU5ODNiMTg2MWZjZjRiN2FkYWQyMDc1MDBkZmE3MWY
1Yzg3MzcwNWYpdAAQYnJvd3NlclVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IExpbnV4IHg4Nl82NC
kgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhc
mkvNTM3LjM2dAANdGVzdEFjY291bnRJZHQAJDExMjE1NmI3LTU2OTMtNDI3NC04ZWJjLTVlZmMwYTk5OGY0
ZnQABW9yZ0lkdAAQODUxNzE3NDAxOTUyMzcwNHQACXVzZXJBZ2VudHQAZU1vemlsbGEvNS4wIChYMTE7IEx
pbnV4IHg4Nl82NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS
4wLjAuMCBTYWZhcmkvNTM3LjM2dAAGcGFnZUlkdAAPd2ViYXBwLm5vdGVib29rdAAJY2x1c3RlcklkdAAUM
TAxOS0wOTA3NTYtb2NpdHkwd2J0AA13b3JrbG9hZENsYXNzdAAKY29tbWFuZFJ1bnQADXNlcnZlckV2ZW50
SWR0ACxDZ3dJcGFPUnZRWVEyUGI5M2dJNkVFOEV0OE9hV1VabGx6d2ZvNlI1WXk0PXQACHJvb3RPcElkdAA
jV2ViYXBwQXJtZXJpYVByb3h5LTkzMTRjOThjMzNjYTdkMWJ0AA9zcGFya0pvYkdyb3VwSWR0AEg1NDAwMD
Y1Mjg1NTg1MDg3NTkwXzcyNzEzMzI1NTkwMzMwMDQzNTdfM2EyNzJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiN
DY2Yjl0AApwYWdlVmlld0lkdAAVMTczODgxOTE2MDQ2MDdtNXNqNHZkdAAGcmVwbElkdAATNTQwMDA2NTI4
NTU4NTA4NzU5MHQACXNlc3Npb25JZHQAMGF1dGgtN2Q0OGRmOTdmNS1iY2hqNi0xNzdpcXZ0ZXVqZGJ1MWc
wYWs0c2VpZDZvMnQADmNsdXN0ZXJDcmVhdG9ydAAQY19qb2huLmZAZ21nLmNvbXQAE29yaWdpbmF0ZWRGcm
9tRW52b3l0AAR0cnVldAAUbnVtRXhlY3V0aW9uQ29tbWFuZHN0AAExdAAQY2xpZW50QnJhbmNoTmFtZXQAC
kRFUFJFQ0FURUR0AAp3b3JrbG
9hZElkdAAkOTMxNzkyY2MtMGI0ZS00NzJjLTlmM2YtNmUxNTFlMzY3OGE0dAAPY2xpZW50VGltZXN0YW1wd
AANMTczODgyMjA1MzQ5NnQAC2NsdXN0ZXJUeXBldAAEc3BvdHQACXJlcXVlc3RJZHQAJDE5OTYyNWYyLTIz
ZDUtNDkyZS04YTU3LTgyZmEwMDEzMWUzNHQAD2Jyb3dzZXJIYXNGb2N1c3QABHRydWV0AA9jb21tYW5kTGF
uZ3VhZ2V0AAZweXRob250AAljb21tYW5kSWR0ABM3MjcxMzMyNTU5MDMzMDA0MzU3dAAPcXVlcnlQYXJhbW
V0ZXJzdAATP289ODUxNzE3NDAxOTUyMzcwNHQABnVzZXJJZHQAEDc1MzM5ODA4OTAxNzU5NTJ0AAl0ZXN0T
3JnSWR0ABA4NTE3MTc0MDE5NTIzNzA0dAAPYnJvd3NlcklzSGlkZGVudAAFZmFsc2V0AAxjbGllbnRMb2Nh
bGV0AAJlbnQACmJyYW5jaE5hbWV0AEZ3ZWJhcHBfMjAyNS0wMS0zMV8wMS4xOC4zNlpfd2ViYXBwLWhvdGZ
peC0yMDI1MDEyNF9mZDE4Yjk4N18xODA2NTY2MDk3dAAGb3BUeXBldAAGRHJpdmVydAAKdGVzdFVzZXJJZH
QAEDc1MzM5ODA4OTAxNzU5NTJ0ABBzb3VyY2VQb3J0TnVtYmVydAABMHQAC2V4ZWN1dGlvbklkdAAgM2EyN
zJhNDc3MDZhNDBkOTgzZjk5MmY2M2FiNDY2Yjl0AAR1c2VydAAPY19kZXYuY0BnbWcuY29tdAAPYnJvd3Nl
ckhvc3ROYW1ldAAmZGJjLTc0YzI4ZTVlLTI3MmUuY2xvdWQuZGF0YWJyaWNrcy5jb210AApwYXJlbnRPcEl
kdAAXRHJpdmVyLTg3MWY4Mzc4YzQwYTI3N2V0AAxqZXR0eVJwY1R5cGV0ADJJbnRlcm5hbERyaXZlckJhY2
tlbmRNZXNzYWdlcyREcml2ZXJCYWNrZW5kUmVxdWVzdHhzcgArY29tLmRhdGFicmlja3MudW5pdHkuVW5pd
HlDcmVkZW50aWFsTWFuYWdlcmFqmM1oaY8qAgAPWgAGY2xvc2VkWgAVcmVndWxhclJlZnJlc2hFbmFibGVk
SgAXdGVtcENyZWRlbnRpYWxzTWF4Q291bnRKACV0ZW1wQ3JlZGVudGlhbHNSZWZyZXNoVGhyZXNob2xkTWl
sbGlzSgAYdGVtcENyZWRlbnRpYWxzVHRsTWlsbGlzSgAVdG9rZW5Jbml0aWFsVHRsTWlsbGlzRAAVdG9rZW
5SZWZyZXNoVGhyZXNob2xkSgAOdG9rZW5UdGxNaWxsaXNaABN1bml0eVJlZnJlc2hFbmFibGVkTAA2Y29tJ
GRhdGFicmlja3MkdW5pdHkkVW5pdHlDcmVkZW50aWFsTWFuYWdlciQkYXBpVXJsT3B0dAAOTHNjYWxhL09w
dGlvbjtMAEBjb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJCRpbml0UmVndWx
hclRva2VuT3B0cQB+AKNMAD5jb20kZGF0YWJyaWNrcyR1bml0eSRVbml0eUNyZWRlbnRpYWxNYW5hZ2VyJC
Rpbml0VW5pdHlUb2tlbk9wdHEAfgCjTAAVbGF0ZXN0UmVndWxhclRva2VuT3B0cQB+AKNMABNsYXRlc3RVb
ml0eVRva2VuT3B0cQB+AKNMAAdzY29wZUlkcQB+AAR4cAAAAAAAAAAAJxAAAAAAAAST4AAAAAAANu6AAAAA
AAKTLgA/
7MzMzMzMzQAAAAAFJlwAAXNxAH4ADnNyAAxqYXZhLm5ldC5VUkyWJTc2GvzkcgMAB0kACGhhc2hDb2RlSQA
EcG9ydEwACWF1dGhvcml0eXEAfgAETAAEZmlsZXEAfgAETAAEaG9zdHEAfgAETAAIcHJvdG9jb2xxAH4ABE
wAA3JlZnEAfgAEeHD//////////
3QAHGlyZWxhbmQuY2xvdWQuZGF0YWJyaWNrcy5jb21xAH4ALnEAfgCodAAFaHR0cHNweHNxAH4ADnNyACRj
b20uZGF0YWJyaWNrcy51bml0eS5SZWd1bGFyQXBpVG9rZW5sjc7kF2uBfQIAA0oADGV4cGlyeVRpbWVNc0w
AAmlkcQB+AKNMAAV2YWx1ZXEAfgAEeHAAAAGU3HIdbXNyAAtzY2FsYS5Ob25lJEZQJPZTypSsAgAAeHEAfg
APdAAkZGtlYTFmN2JiYjVhYzRjMjc2NGFlOTQzYjE4ZTZhMTc4Y2Q5c3EAfgAOc3IAImNvbS5kYXRhYnJpY
2tzLnVuaXR5LlVuaXR5QXBpVG9rZW6wHqLAOHsNbQIAA0oADGV4cGlyeVRpbWVNc0wAAmlkcQB+AKNMAAV2
YWx1ZXEAfgAEeHAAAAGU3HIdbXEAfgCudAAkZGtlYTVhOTM0M2IwODA4MmJiODBjMjJiOTZmNDA0NDA5NzE
xcQB+AKpzcQB+AA5xAH4AsnQAJGNjNTRhMmYwLWY0OWEtNGFmMC1hNzY5LTc3YjBiNGNlMDkyZnEAfgC1c3
IAKmNvbS5kYXRhYnJpY2tzLnVuaXR5LlVuaXR5UGlwZWxpbmVSZWdpc3RyeVFQOpARQb4YAgABTAALcGlwZ
WxpbmVPcHRxAH4Ao3hwcQB+AK5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVW5pdHlTQU1SZWdpc3RyeWBq
HHckrz9DAgADTAAIcmVhZFNBTXN0ACxMamF2YS91dGlsL2NvbmN1cnJlbnQvQ29uY3VycmVudFNraXBMaXN
0TWFwO0wAB3VjVXRpbHN0AChMY29tL2RhdGFicmlja3MvdW5pdHkvVW5pdHlDYXRhbG9nVXRpbHM7TAAJd3
JpdGVTQU1zcQB+ALl4cHNyACpqYXZhLnV0aWwuY29uY3VycmVudC5Db25jdXJyZW50U2tpcExpc3RNYXCIR
nWuBhFGpwMAAUwACmNvbXBhcmF0b3J0ABZMamF2YS91dGlsL0NvbXBhcmF0b3I7eHBwc3IAI2NvbS5kYXRh
YnJpY2tzLnVuaXR5Lk5vcm1hbGl6ZWRQYXRoEJ5WIgT7QbICAAJMACljb20kZGF0YWJyaWNrcyR1bml0eSR
Ob3JtYWxpemVkUGF0aCQkcGF0aHQAG0xvcmcvYXBhY2hlL2hhZG9vcC9mcy9QYXRoO0wAKGNvbSRkYXRhYn
JpY2tzJHVuaXR5JE5vcm1hbGl6ZWRQYXRoJCRzdHJxAH4ABHhwc3IAGW9yZy5hcGFjaGUuaGFkb29wLmZzL
lBhdGgAAAAAAArQDwIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOe
SasDAAFMAAZzdHJpbmdxAH4ABHhwdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1
tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMW
Y0ZmYyZGQtMGM0NS00OWYxLWI4YWYtN2RmMDkyYmIyNzI3eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY
2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIw
NTVmZGUvdGFibGVzLzFmNGZmMmRkLTBjNDUtNDlmMS1iOGFmLTdkZjA5MmJiMjcyN3NyACVjb20uZGF0YWJ
yaWNrcy51bml0eS5TQU1XaXRoVGltZXN0YW1wcfAbCWEa96cCAAJKAAl0aW1lc3RhbXBMAANzYW10ABpMY2
9tL2RhdGFicmlja3MvdW5pdHkvU0FNO3hwAAAAeYqc+chzcgAdY29tLmRhdGFicmlja3MudW5pdHkuVGFib
GVTQU3FevgDccbhAgIABEwAC2FjY2Vzc1BvaW50cQB+AKNMABFlbmNyeXB0aW9uRGV0YWlsc3EAfgCjTAAC
aWR0ABBMamF2YS91dGlsL1VVSUQ7TAAIcGFyZW50SWRxAH4Ao3hwcQB+AK5xAH4ArnNyAA5qYXZhLnV0aWw
uVVVJRLyZA/eYbYUvAgACSgAMbGVhc3RTaWdCaXRzSgALbW9zdFNpZ0JpdHN4cLivffCSuycnH0/
y3QxFSfFzcQB+AA5zcgAlY29tLmRhdGFicmlja3MudW5pdHkuVGFibGVEZXBlbmRlbnRJZLMLrrOVi2ZZAg
ABTAACaWRxAH4AzXhwc3EAfgDPhh8SlPc81Ap+8sad1ptCEnNxAH4Av3NxAH4AwnNxAH4AxXQAh3MzOi8vZ
GF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZjJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIy
LTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFibGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY
5NGU5M3h0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZX
QvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy8yMWEyM2QwNC1jMTc3LTQ5Z
DUtOGQ3Ny0xOGRhNGRmOTRlOTNzcQB+AMkAAAB5hZLwYnNxAH4AzHEAfgCucQB+AK5zcQB+AM+NdxjaTflO
kyGiPQTBd0nVcQB+AK5zcQB+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN
0YWNrLWYyYzY1LW1ldGFzdG9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1Zm
RlL3RhYmxlcy80MjhmMTViZS03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY
2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05
MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3E
AfgDJAAAAeYWLLYVzcQB+AMxxAH4ArnEAfgCuc3EAfgDPo11eLCyaep5CjxW+cRlEF3EAfgCuc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvOTE0M2E1ZmEtOT
E2Yy00NjFhLTg3MGQtM2ZiM2I5MTYwZmE0eHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzkxNDNhNWZhLTkxNmMtNDYxYS04NzBkLTNmYjNiOTE2MGZhNHNxAH4AyQAAAHmOlBJYc3EAfgDMcQB
+AK5xAH4ArnNxAH4Az4cNP7O5Fg+kkUOl+pFsRhpzcQB+AA5zcQB+ANJzcQB+AM+3fpnn8cX9jKwd+PjiwE
yIcHhzcgA3Y29tLmRhdGFicmlja3Muc3FsLm1hbmFnZWRjYXRhbG9nLlVuaXR5Q2F0YWxvZ1V0aWxzSW1wb
OlxCzQ+WTLyAgAAeHBzcQB+ALxwc3EAfgC/
c3EAfgDCc3EAfgDFdACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFjay1mMmM2NS1tZXRhc3RvcmU
tYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS90YWJsZXMvMjFhMjNkMDQtYz
E3Ny00OWQ1LThkNzctMThkYTRkZjk0ZTkzeHQAh3MzOi8vZGF0YWJyaWNrcy13b3Jrc3BhY2Utc3RhY2stZ
jJjNjUtbWV0YXN0b3JlLWJ1Y2tldC83ZGMxZTQ5Yy0yZGIyLTQyZmUtOTFjOS0xZTc4YWIwNTVmZGUvdGFi
bGVzLzIxYTIzZDA0LWMxNzctNDlkNS04ZDc3LTE4ZGE0ZGY5NGU5M3NxAH4AyQAAAHmFlqIAcQB+ANtzcQB
+AL9zcQB+AMJzcQB+AMV0AIdzMzovL2RhdGFicmlja3Mtd29ya3NwYWNlLXN0YWNrLWYyYzY1LW1ldGFzdG
9yZS1idWNrZXQvN2RjMWU0OWMtMmRiMi00MmZlLTkxYzktMWU3OGFiMDU1ZmRlL3RhYmxlcy80MjhmMTViZ
S03MTE5LTQ0MTctYTM1ZC01ZTJjMmM5YTdhOWV4dACHczM6Ly9kYXRhYnJpY2tzLXdvcmtzcGFjZS1zdGFj
ay1mMmM2NS1tZXRhc3RvcmUtYnVja2V0LzdkYzFlNDljLTJkYjItNDJmZS05MWM5LTFlNzhhYjA1NWZkZS9
0YWJsZXMvNDI4ZjE1YmUtNzExOS00NDE3LWEzNWQtNWUyYzJjOWE3YTllc3EAfgDJAAAAeYWPskRxAH4A43
B4, spark.hadoop.fs.adl.impl.disable.cache=true,
spark.hadoop.fs.s3n.impl.disable.cache=true,
spark.databricks.acl.dfAclsEnabled=true,
spark.databricks.cloudfetch.hasRegionSupport=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2S3=0,
spark.hadoop.fs.s3a.fast.upload.active.blocks=32,
spark.hadoop.fs.s3.impl.disable.cache=true,
spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperB
ridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false,
spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=,
spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=default,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=1,
spark.databricks.replId=ReplId-4af0e-2c437-37ac6-6,
spark.databricks.pyspark.enablePy4JSecurity=true,
spark.r.backendConnectionTimeout=604800,
spark.databricks.unityCatalog.queryFederation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2File=0,
spark.databricks.clusterUsageTags.clusterEbsVolumeType=GENERAL_PURPOSE_SSD,
spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5,
spark.databricks.clusterUsageTags.driverNodeType=m5d.xlarge,
spark.sql.catalogImplementation=hive,
spark.databricks.clusterUsageTags.clusterGeneration=1468,
spark.databricks.clusterUsageTags.clusterMaxWorkers=5,
spark.databricks.wsfs.workspacePrivatePreview=true,
spark.databricks.clusterUsageTags.isSingleUserCluster=false, [email protected],
spark.databricks.connector.mssqlserver.driverVersion=current,
spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.dae
mon.data.client.adl.AdlCredentialContextTokenProvider,
spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.datab
ricks.unity.TokenServiceApiTokenProvider,
spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebapp
ScimClient, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false,
spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true,
spark.hadoop.parquet.block.size.row.check.min=10,
spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils,
spark.databricks.enablePublicDbfsFuse=false,
spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.mlflow.autologging.enableGenAIFlavors=true,
spark.hadoop.fs.wasb.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.cloudProvider=AWS,
spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false,
spark.hadoop.hive.server2.thrift.http.port=10000, spark.shuffle.manager=SORT,
spark.hadoop.hive.server2.keystore.password=[REDACTED],
spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider,
spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30,
spark.hadoop.parquet.abfs.readahead.optimization.enabled=true,
spark.databricks.clusterUsageTags.enableElasticDisk=true, orgId=8517174019523704,
spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DB
CEventLoggingListener,
spark.databricks.clusterUsageTags.clusterStateMessage=Starting
Spark, spark.databricks.clusterUsageTags.sparkVersion=13.3.x-photon-scala2.12,
spark.databricks.clusterUsageTags.hailEnabled=false,
spark.databricks.unityCatalog.legacy.enableCrossScopeCredCache=true,
spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.D
atabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/hadoop-
safety-jars/*:/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/
jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*,
spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=0,
spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=100,
spark.databricks.acl.secureCatalogBatchFilterDatabases=true,
spark.hadoop.fs.wasbs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSy
stem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false,
spark.databricks.pyspark.py4jConfRedaction.enabled=true,
spark.databricks.unityCatalog.volumes.fuse.server.enabled=true,
spark.speculation.multiplier=3,
spark.databricks.delta.liquid.fixInvalidStateAfterPhaseOut=true,
spark.databricks.clusterUsageTags.clusterPythonVersion=3,
spark.databricks.clusterUsageTags.ngrokNpipEnabled=true,
spark.databricks.unityCatalog.glue.federation.enabled=true,
spark.databricks.pyspark.enableIptables=true,
spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false,
spark.databricks.pipelines.analysisId=DLTAnalysisID-3f0c5-e7724-98e0b-e,
spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1,
spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(re
dacted), spark.databricks.unityCatalog.enforce.permissions=true,
spark.hadoop.hive.server2.idle.operation.timeout=7200000,
spark.hadoop.hive.server2.transport.mode=http,
spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30,
spark.databricks.acl.enabled=true,
spark.databricks.notebook.path=/Users/[email protected]/O9/O9_Sales/
BPP_SALES_SPORTS/(Clone) BPP_Sports_Sales_Till_29Feb2024_Jan_19_2025,
spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLo
gStore, spark.hadoop.fs.s3a.fast.upload.default=true,
spark.databricks.driver.preferredMavenCentralMirrorUrl=https://maven-
central.storage-download.googleapis.com/maven2/,
spark.master=spark://10.183.232.44:7077,
spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false,
spark.databricks.execution.rootSqlExecutionId=32,
spark.files.fetchFailure.unRegisterOutputOnHost=true,
spark.hadoop.fs.s3a.retry.throttle.interval=500ms,
spark.sql.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.privateLinkEnabled=false, PYTHON_UID=1003,
spark.databricks.pyspark.runAsLowPrivilegeUser=true,
spark.hadoop.fs.s3a.fast.upload=true,
spark.databricks.secret.sparkConf.keys.toRedact=,
spark.databricks.pyspark.enableProcessIsolation=true,
NFS_REPL_TEMP_DIR=/local_disk0/.ephemeral_nfs/repl_tmp_data/ReplId-4af0e-2c437-
37ac6-6, spark.databricks.clusterUsageTags.attribute_tag_dust_resource_class=,
spark.executor.memory=5397m, spark.hadoop.fs.azure.user.agent.prefix=,
spark.executor.id=driver,
spark.databricks.unityCatalog.userIsolation.scala.enabled=true,
spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.dae
mon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms,
spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED],
spark.databricks.clusterUsageTags.clusterLogDestination=,
spark.databricks.clusterUsageTags.shardName=ireland-prod,
spark.scheduler.listenerbus.eventqueue.capacity=20000,
spark.hadoop.databricks.loki.fileSystemCache.enabled=true,
spark.databricks.env=databricks-python-repl-ipykernel,
spark.databricks.sql.ES1163306=false,
spark.databricks.unityCatalog.externalLocationFallbackMode.enabled=false,
spark.hadoop.parquet.page.size.check.estimate=false,
spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-
keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true,
spark.databricks.wsfsPublicPreview=true, spark.rdd.compress=true,
spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.clien
t.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000,
spark.databricks.driverNodeTypeId=m5d.xlarge,
spark.databricks.hive.acl.checkPermission.impl=com.databricks.sql.acl.HiveCheckPerm
issions, spark.hadoop.fs.azure.skip.metrics=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Workspace=0,
spark.databricks.clusterUsageTags.attribute_tag_platform_name=,
spark.databricks.sql.jdbcDialectIgnoreQueriesWithForbiddenKeywords=false,
spark.hadoop.hive.hmshandler.retry.attempts=10,
spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-
wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.attribute_tag_dust_runbot_id=,
spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216,
spark.databricks.acl.secureCatalogRestrictObjLists=true,
spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-
Token:spark.databricks.token,X-Databricks-Non-UC-User-
Token:spark.databricks.non.uc.token,X-Databricks-Api-
Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-
Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-
Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-
Token:spark.databricks.synapse.token,X-Databricks-AWS-
Credentials:spark.databricks.aws.creds,X-Databricks-User-
Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-
Databricks-Oauth-Identity-Custom-
Claim:spark.databricks.oauthCustomIdentityClaims,X-Databricks-Workload-
Id:spark.databricks.workload.id,X-Databricks-Workload-
Class:spark.databricks.workload.name, spark.databricks.automl.serviceEnabled=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=false,
spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.data
bricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider,
spark.akka.frameSize=256, spark.ui.killEnabled=false,
spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daem
on.driver.aws.AwsCredentialContextTokenProvider,
spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-
s3n.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=,
spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoo
p.fs.gcs.GoogleHadoopFS,
spark.databricks.clusterUsageTags.sparkImageLabel=release__13.3.x-snapshot-photon-
scala2.12__databricks__13.3.30__676124b__5fd83ac__jenkins__61feefd__format-3,
spark.serializer.objectStreamReset=100,
spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false,
spark.sparkr.use.daemon=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Abfss=0,
spark.databricks.clusterUsageTags.currentAttemptContainerZoneId=eu-west-1a,
spark.databricks.clusterUsageTags.clusterFirstOnDemand=2, PYTHON_USERNAME=spark-
46b4cba4-6e97-4786-9ef9-da, spark.files.useFetchCache=false,
spark.sql.hive.metastore.version=0.13.0,
spark.databricks.clusterUsageTags.clusterNumCustomTags=1,
spark.databricks.clusterUsageTags.clusterName=Unity Catalog Cluster,
spark.databricks.hadoop.disableLocalFileSystem=true,
spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgre
sql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.da
tabricks,com.codahale,com.fasterxml.jackson,shaded.databricks,grpc_shaded.com.datab
ricks.api.proto.managedcatalog,
spark.databricks.clusterUsageTags.clusterNumSshKeys=0,
spark.hadoop.fs.s3a.attempts.maximum=10,
spark.databricks.clusterUsageTags.orgId=8517174019523704,
spark.memory.offHeap.enabled=true,
spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy
, spark.hadoop.fs.s3a.block.size=67108864,
spark.databricks.notebook.id=589147838710961,
spark.hadoop.hive.server2.enable.doAs=false,
spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false,
accountId=112156b7-5693-4274-8ebc-5efc0a998f4f,
spark.databricks.clusterUsageTags.clusterResourceClass=default,
spark.hadoop.hive.server2.idle.session.timeout=900000,
spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-
abfss.impl.disable.cache=true,
spark.databricks.clusterUsageTags.autoTerminationMinutes=20,
spark.databricks.clusterUsageTags.attribute_tag_dust_suite=,
spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.ba
ckend.daemon.driver.credentials.GlueClientExecutorServiceFactory,
spark.databricks.clusterUsageTags.enableDfAcls=true,
spark.databricks.clusterUsageTags.instanceProfileUsed=false,
spark.hadoop.parquet.block.size.row.check.max=10,
spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.183.251.100,
spark.databricks.safespark.externalUDF.plan.limit=128,
spark.databricks.clusterSource=UI, spark.hadoop.fs.cpfs-
abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-
adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.region=eu-west-1, spark.ui.port=40001,
spark.databricks.acl.defaultSessionCatalogOnly=true,
spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=dbc-
74c28e5e-272e.cloud.databricks.com, spark.hadoop.fs.abfss.impl.disable.cache=true,
spark.hadoop.parquet.filter.columnindex.enabled=false,
spark.databricks.sql.functions.aiGen.endpointName=databricks-meta-llama-3-1-70b-
instruct-dbr, spark.sql.execution.id=32,
spark.databricks.acl.filesystem.checkPermission.impl=com.databricks.sql.acl.CheckFi
leSystemSafety,
spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backen
d.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider,
spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-
abfss.impl.disable.cache=true,
spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem,
spark.databricks.acl.secureCatalogBatchFilterTables=true,
spark.databricks.unityCatalog.lakehouseFederation.writes.enabled=true,
spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient,
spark.databricks.clusterUsageTags.dataPlaneRegion=eu-west-1,
spark.databricks.clusterUsageTags.clusterOwnerUserId=1687111647522361,
spark.streaming.driver.writeAheadLog.allowBatching=true,
spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false,
spark.hadoop.fs.azure.authorization.caching.enable=false,
spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.back
end.daemon.driver.credentials.CredentialScopeS3TokenProvider,
spark.databricks.unityCatalog.hms.federation.enableDbfsSupport=true,
spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136,
spark.repl.class.uri=spark://10.183.232.44:36145/classes,
spark.databricks.acl.skipCheckingPlans=NONE,
spark.databricks.dynamicConf.demoFlag=10,
spark.databricks.deltaSharing.clientClassName=com.databricks.deltasharing.DataShari
ngClientImpl,
spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datas
ources.SQLHadoopMapReduceCommitProtocol,
spark.databricks.optimizer.optimizeLimit.maxLimit=10001,
spark.driver.host=10.183.232.44, spark.driver.allowMultipleContexts=false,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Gcs=0,
spark.databricks.driverNfs.enabled=true,
spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-
abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.hadoop.fs.s3a.multipart.size=10485760,
spark.databricks.acl.secureDefaultSessionCatalogImpls=com.databricks.sql.transactio
n.tahoe.catalog.DeltaCatalog,
spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-
s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.task.reaper.enabled=true,
spark.databricks.clusterUsageTags.attribute_tag_dust_runner=,
spark.databricks.clusterUsageTags.clusterNodeType=r6i.xlarge,
spark.databricks.cloudProvider=AWS,
spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8517174019523704-
0ca02371-3c88-4d34-8f21-7e85514035f6,
spark.databricks.inherited.credentials.keys.spark.databricks.uc.api.token=*********
(redacted), spark.databricks.sessionUUID=46b4cba4-6e97-4786-9ef9-dab57a49e591,
spark.shuffle.memoryFraction=0.2,
spark.databricks.clusterUsageTags.clusterScalingType=autoscaling,
spark.hadoop.fs.fcfs-s3.impl.disable.cache=true,
spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1
,
spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedac
torProxyImpl, spark.databricks.clusterUsageTags.clusterEbsVolumeSize=100,
spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false,
spark.app.name=Databricks Shell,
spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU,
spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheI
nvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties,
spark.databricks.clusterUsageTags.containerZoneId=auto,
spark.databricks.workload.name=commandRun, spark.sql.streaming.stopTimeout=15s,
spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.
hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider,
spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentials
Provider, spark.databricks.acl.secureCatalogCheckUsageOnCatalog=true,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false,
spark.files.overwrite=true,
spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore,
spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScope
FileSystem,
spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=fals
e,
spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.ba
ckend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory,
spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true,
spark.hadoop.fs.gs.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3,
spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false,
spark.shuffle.reduceLocality.enabled=false,
spark.databricks.repl.enableClassFileCleanup=true,
spark.databricks.clusterUsageTags.awsWorkspaceIMDSV2EnablementStatus=false,
spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-
abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-
backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true,
spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false,
spark.hadoop.databricks.fs.perfMetrics.enable=true,
spark.databricks.eventLog.enabled=true, spark.databricks.isShieldWorkspace=false,
spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000,
spark.databricks.api.url=https://ireland.cloud.databricks.com,
spark.databricks.inherited.spark.databricks.api.url=https://ireland.cloud.databrick
s.com, spark.hadoop.fs.fcfs-
wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem,
spark.databricks.clusterUsageTags.clusterPinned=true,
spark.speculation.quantile=0.9, spark.hadoop.fs.cpfs-
s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
spark.hadoop.fs.cpfs-
adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem,
userId=7533980890175952, spark.hadoop.fs.wasb.impl.disable.cache=true,
spark.sql.legacy.createHiveTableByDefault=false,
spark.databricks.passthrough.enabled=false,
spark.databricks.clusterUsageTags.instanceBootstrapType=vm-selfbootstrap,
spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048,
spark.databricks.credential.scope.fs.wasbs.tokenProviderClassName=com.databricks.ba
ckend.daemon.driver.credentials.CredentialScopeWASBTokenProvider,
spark.databricks.qpl.queryId=3bac0afb-853f-45e7-93ba-6a407147729c,
spark.databricks.clusterUsageTags.isIMv2Enabled=true, spark.hadoop.fs.fcfs-
wasb.impl.disable.cache=true,
spark.hadoop.databricks.s3.verifyBucketExists.enabled=false,
spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs,
spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.Dire
ctoryAtomicCommitProtocol,
spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem,
spark.databricks.unityCatalog.enableLibrariesApiOnSharedCluster=false,
spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true,
spark.connect.extensions.relation.classes=io.delta.connect.DeltaRelationPlugin,
spark.scheduler.pool=5400065285585087590,
spark.hadoop.databricks.loki.fileStatusCache.enabled=true,
spark.databricks.pyspark.iptable.outbound.whitelisted.ports=80,
spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true,
spark.databricks.ui.logViewingEnabled=false, spark.memory.offHeap.size=16977494016,
spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false,
spark.databricks.clusterUsageTags.clusterMinWorkers=2,
spark.databricks.driverNfs.clusterWideVirtualEnv=/local_disk0/.ephemeral_nfs/
cluster_libraries/python,
spark.databricks.clusterUsageTags.clusterState=Restarting,
spark.sql.parquet.cacheMetadata=true,
spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER,
spark.databricks.clusterUsageTags.effectiveSparkVersion=13.3.x-photon-scala2.12,
spark.databricks.unityCatalog.hms.federation.enabled=true,
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Dbfs=0,
spark.databricks.clusterUsageTags.driverContainerId=6fdcd158672c4bcf8f2b29d7fc0fba8
6,
spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.da
emon.driver.NephosDbfsReadinessCheckClient,
spark.databricks.clusterUsageTags.clusterUnityCatalogMode=USER_ISOLATION,
spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"interactive_cluster_tag",
"value":"dev_unity_catalog_cluster"},{"key":"Vendor","value":"Databricks"},
{"key":"Creator","value":"[email protected]"},{"key":"ClusterName","value":"Unity
Catalog Cluster"},{"key":"ClusterId","value":"1019-090756-ocity0wb"},
{"key":"Name","value":"workerenv-8517174019523704-0ca02371-3c88-4d34-8f21-
7e85514035f6-worker"}],
spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2Volumes=0,
spark.hadoop.parquet.page.write-checksum.enabled=true,
spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin,com.databricks.spark.
connect.LocalSparkConnectPlugin, spark.cleaner.referenceTracking.blocking=false,
spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-
s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.attribute_tag_service=,
spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.
DataDaemonCloudPresignedUrlRequester,
spark.databricks.clusterUsageTags.clusterOwnerOrgId=8517174019523704,
spark.jobGroup.id=5400065285585087590_7271332559033004357_3a272a47706a40d983f992f63
ab466b9, spark.databricks.workload.id=931792cc-0b4e-472c-9f3f-6e151e3678a4,
spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServe
rDBUtils, spark.databricks.dataLineage.mergeIntoV2Enabled=false,
spark.hadoop.fs.s3a.impl.disable.cache=true,
spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false,
spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSec
retKeyRedactorProxy}) by listener SQLAppStatusListener took 11.022657064s.
25/02/06 06:22:59 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:22:59 INFO TaskSetManager: Finished task 17.0 in stage 464.0 (TID 1995)
in 3137 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:22:59 INFO TaskSetManager: Finished task 15.0 in stage 464.0 (TID 1993)
in 3155 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:22:59 INFO TaskSetManager: Finished task 13.0 in stage 464.0 (TID 1991)
in 3156 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:22:59 INFO TaskSetManager: Finished task 11.0 in stage 464.0 (TID 1989)
in 3156 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:22:59 INFO TaskSetManager: Starting task 0.0 in stage 466.0 (TID 1996)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:22:59 INFO TaskSetManager: Starting task 1.0 in stage 466.0 (TID 1997)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:22:59 INFO TaskSetManager: Starting task 2.0 in stage 466.0 (TID 1998)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:22:59 INFO TaskSetManager: Starting task 3.0 in stage 466.0 (TID 1999)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:22:59 INFO TaskSetManager: Starting task 4.0 in stage 466.0 (TID 2000)
(10.183.229.123, executor 0, partition 4, PROCESS_LOCAL,
25/02/06 06:22:59 INFO TaskSetManager: Starting task 5.0 in stage 466.0 (TID 2001)
(10.183.229.123, executor 0, partition 5, PROCESS_LOCAL,
25/02/06 06:22:59 INFO MemoryStore: Block broadcast_115 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:23:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:23:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:23:09 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0,
New Ema: 1.0
25/02/06 06:23:09 ERROR DriverCorral: Invalid executionContextId:
MessagePacket(jupyter_message,Metadata(None,Some(kernel_info_request),Some(11385820
23817806),Some(1019-090756-
ocity0wb),Some(237442136387608794),None,Some(CommandContext(...)),None),MessagePayl
oad(Map(),Some(control),Some(Header(Some(kernel_info_request),Some(1d6bcbb4-8499-
40d4-bf53-75abca143292),Some(databricksnbid:1138582023817806),Some(),Some(2025-02-
06T06:23:01.812Z),Some(5.4))),None,Map(),List()),null)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 16.0 in stage 463.0 (TID 1976)
in 13642 ms on 10.183.229.123 (executor 0) (17/18)
25/02/06 06:23:09 INFO DriverCorral: Starting sql repl ReplId-661af-0adf4-bffc6-b
25/02/06 06:23:09 INFO TaskSetManager: Finished task 17.0 in stage 463.0 (TID 1977)
in 13643 ms on 10.183.229.123 (executor 0) (18/18)
25/02/06 06:23:09 INFO TaskSchedulerImpl: Removed TaskSet 463.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:23:09 INFO DAGScheduler: ShuffleMapStage 463
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
239.423 s
25/02/06 06:23:09 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:23:09 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 565,
ShuffleMapStage 544, ShuffleMapStage 646, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 613,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 580, ShuffleMapStage 624, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 464, ShuffleMapStage 508,
ShuffleMapStage 487)
25/02/06 06:23:09 INFO DAGScheduler: waiting: Set()
25/02/06 06:23:09 INFO DAGScheduler: failed: Set()
25/02/06 06:23:09 INFO MemoryStore: Block broadcast_115_piece0 stored as bytes in
memory (estimated size 126.0 KiB, free 20.2 GiB)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 2.0 in stage 464.0 (TID 1980)
in 13661 ms on 10.183.232.45 (executor 2) (5/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 5.0 in stage 464.0 (TID 1983)
in 13658 ms on 10.183.232.45 (executor 2) (6/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 4.0 in stage 464.0 (TID 1982)
in 13662 ms on 10.183.232.45 (executor 2) (7/18)
25/02/06 06:23:09 INFO TaskSetManager: Starting task 6.0 in stage 466.0 (TID 2002)
(10.183.232.45, executor 2, partition 6, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 7.0 in stage 466.0 (TID 2003)
(10.183.232.45, executor 2, partition 7, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 8.0 in stage 466.0 (TID 2004)
(10.183.232.45, executor 2, partition 8, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 9.0 in stage 466.0 (TID 2005)
(10.183.232.45, executor 2, partition 9, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 10.0 in stage 466.0 (TID 2006)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 11.0 in stage 466.0 (TID 2007)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 12.0 in stage 466.0 (TID 2008)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 13.0 in stage 466.0 (TID 2009)
(10.183.233.191, executor 5, partition 13, PROCESS_LOCAL,
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.232.44:37705 (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 0.0 in stage 464.0 (TID 1978)
in 13669 ms on 10.183.229.123 (executor 0) (8/18)
25/02/06 06:23:09 INFO SparkContext: Created broadcast 115 from broadcast at
TaskSetManager.scala:711
25/02/06 06:23:09 INFO TaskSetManager: Finished task 8.0 in stage 464.0 (TID 1986)
in 13665 ms on 10.183.233.191 (executor 5) (9/18)
25/02/06 06:23:09 INFO BlockManagerInfo: Removed broadcast_112_piece0 on
10.183.246.72:43231 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Removed broadcast_112_piece0 on
10.183.249.197:40023 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Removed broadcast_112_piece0 on
10.183.232.45:45579 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 1.0 in stage 464.0 (TID 1979)
in 13673 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 3.0 in stage 464.0 (TID 1981)
in 13672 ms on 10.183.232.45 (executor 2) (11/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 7.0 in stage 464.0 (TID 1985)
in 13667 ms on 10.183.233.191 (executor 5) (12/18)
25/02/06 06:23:09 INFO TaskSetManager: Starting task 14.0 in stage 466.0 (TID 2010)
(10.183.233.191, executor 5, partition 14, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Finished task 9.0 in stage 464.0 (TID 1987)
in 13668 ms on 10.183.233.191 (executor 5) (13/18)
25/02/06 06:23:09 INFO BlockManagerInfo: Removed broadcast_112_piece0 on
10.183.233.191:42923 in memory (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 6.0 in stage 464.0 (TID 1984)
in 13673 ms on 10.183.233.191 (executor 5) (14/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 16.0 in stage 464.0 (TID 1994)
in 13644 ms on 10.183.249.197 (executor 7) (15/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 12.0 in stage 464.0 (TID 1990)
in 13645 ms on 10.183.249.197 (executor 7) (16/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 14.0 in stage 464.0 (TID 1992)
in 13646 ms on 10.183.249.197 (executor 7) (17/18)
25/02/06 06:23:09 INFO TaskSetManager: Finished task 10.0 in stage 464.0 (TID 1988)
in 13647 ms on 10.183.249.197 (executor 7) (18/18)
25/02/06 06:23:09 INFO TaskSchedulerImpl: Removed TaskSet 464.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:23:09 INFO TaskSetManager: Starting task 15.0 in stage 466.0 (TID 2011)
(10.183.249.197, executor 7, partition 15, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 16.0 in stage 466.0 (TID 2012)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 17.0 in stage 466.0 (TID 2013)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 0.0 in stage 465.0 (TID 2014)
(10.183.249.197, executor 7, partition 0, PROCESS_LOCAL,
25/02/06 06:23:09 INFO TaskSetManager: Starting task 1.0 in stage 465.0 (TID 2015)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:23:09 INFO DAGScheduler: ShuffleMapStage 464
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
239.448 s
25/02/06 06:23:09 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:23:09 INFO DAGScheduler: running: Set(ShuffleMapStage 466,
ShuffleMapStage 481, ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577,
ShuffleMapStage 556, ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529,
ShuffleMapStage 602, ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502,
ShuffleMapStage 517, ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467,
ShuffleMapStage 490, ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 565,
ShuffleMapStage 544, ShuffleMapStage 646, ShuffleMapStage 465, ShuffleMapStage 559,
ShuffleMapStage 538, ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 613,
ShuffleMapStage 505, ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478,
ShuffleMapStage 580, ShuffleMapStage 624, ShuffleMapStage 574, ShuffleMapStage 553,
ShuffleMapStage 568, ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468,
ShuffleMapStage 591, ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470,
ShuffleMapStage 514, ShuffleMapStage 493, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:23:09 INFO DAGScheduler: waiting: Set()
25/02/06 06:23:09 INFO DAGScheduler: failed: Set()
25/02/06 06:23:09 INFO MemoryStore: Block broadcast_116 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:23:09 INFO DriverCorral: Starting sql repl ReplId-5ea28-6c7f2-160b6-6
25/02/06 06:23:09 INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in
memory (estimated size 126.2 KiB, free 20.2 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Removed broadcast_112_piece0 on
10.183.232.44:37705 in memory (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on
10.183.232.44:37705 (size: 126.2 KiB, free: 20.2 GiB)
25/02/06 06:23:09 INFO SparkContext: Created broadcast 116 from broadcast at
TaskSetManager.scala:711
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.233.191:42923 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.246.72:43231 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO DriverCorral: Starting sql repl ReplId-2ad3d-cb3a1-348df-3
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.229.123:38441 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.232.45:45579 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on
10.183.249.197:40023 (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on
10.183.249.197:40023 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:33.
Current active queries:2
25/02/06 06:23:09 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:34.
Current active queries:3
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 8, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:09 INFO BaselineLogicalPlanStats: Setting BaselineLogicalPlanStats
visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ExperimentalLogicalPlanStats: Setting
ExperimentalLogicalPlanStats visitors to
List((NonEquiJoinCondition,com.databricks.sql.optimizer.statsEstimation.experiments
.NonEquiJoinConditionLogicalPlanStatsVisitor$@33a2586c),
(WindowStats,com.databricks.sql.optimizer.statsEstimation.experiments.WindowStatsLo
gicalPlanStatsVisitor$@4cd787c9),
(JoinSelectivityOne,com.databricks.sql.optimizer.statsEstimation.experiments.JoinSe
lectivityOneExperimentalVisitor$@264804f5),
(JoinSelectivityOneGuard,com.databricks.sql.optimizer.statsEstimation.experiments.J
oinSelectivityOneGuardExperimentalVisitor$@1c46de4f),
(LimitStats,com.databricks.sql.optimizer.statsEstimation.experiments.LimitStatsLogi
calPlanStatsVisitor$@3d069493),
(PropagateStatsJoinedExpressions,com.databricks.sql.optimizer.statsEstimation.exper
iments.PropagateStatsForJoinedExpressionsExperimentalVisitor$@4d9fa535))
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:35.
Current active queries:4
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:09 INFO deprecation: fs.default.name is deprecated. Instead, use
fs.defaultFS
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar at
(spark://10.183.232.44:36145/jars/com_databricks_spark_xml_2_12_0_16_0.jar,Some(/
local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:33.
Current active queries:3
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:36.
Current active queries:4
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar at
(spark://10.183.232.44:36145/jars/com_databricks_spark_xml_2_12_0_16_0.jar,Some(/
local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:34.
Current active queries:3
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar at
(spark://10.183.232.44:36145/jars/com_databricks_spark_xml_2_12_0_16_0.jar,Some(/
local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:35.
Current active queries:2
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:37.
Current active queries:3
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:38.
Current active queries:4
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar at
(spark://10.183.232.44:36145/jars/com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar,Some(/
local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:36.
Current active queries:3
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:39.
Current active queries:4
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar at
(spark://10.183.232.44:36145/jars/com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar,Some(/
local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:37.
Current active queries:3
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar at
(spark://10.183.232.44:36145/jars/com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar,Some(/
local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:38.
Current active queries:2
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:40.
Current active queries:3
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:41.
Current active queries:4
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar,
Some(/local_disk0/tmp/
addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:39.
Current active queries:3
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar,
Some(/local_disk0/tmp/
addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:40.
Current active queries:2
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:42.
Current active queries:3
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar,
Some(/local_disk0/tmp/
addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:41.
Current active queries:2
25/02/06 06:23:09 INFO ClusterLoadMonitor: Added query with execution ID:43.
Current active queries:3
25/02/06 06:23:09 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:09 INFO DriverCorral: Starting python repl ReplId-7f315-a9c13-11174-
5
25/02/06 06:23:09 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_math3_3_6_1.jar,Some(/
local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:09 INFO ClusterLoadMonitor: Removed query with execution ID:42.
Current active queries:2
25/02/06 06:23:09 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:44.
Current active queries:3
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:45.
Current active queries:4
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_math3_3_6_1.jar,Some(/
local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:43.
Current active queries:3
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_math3_3_6_1.jar,Some(/
local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:45.
Current active queries:2
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:46.
Current active queries:3
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_collection_compat_2_
12_2_9_0.jar,Some(/local_disk0/tmp/
addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar)) has been added
already. Overwriting of added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:44.
Current active queries:2
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:47.
Current active queries:3
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 3, Current Avg
load: 1, New parallelism: 7
25/02/06 06:23:10 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:10 INFO BaselineLogicalPlanStats: Setting BaselineLogicalPlanStats
visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:10 INFO ExperimentalLogicalPlanStats: Setting
ExperimentalLogicalPlanStats visitors to
List((NonEquiJoinCondition,com.databricks.sql.optimizer.statsEstimation.experiments
.NonEquiJoinConditionLogicalPlanStatsVisitor$@33a2586c),
(WindowStats,com.databricks.sql.optimizer.statsEstimation.experiments.WindowStatsLo
gicalPlanStatsVisitor$@4cd787c9),
(JoinSelectivityOne,com.databricks.sql.optimizer.statsEstimation.experiments.JoinSe
lectivityOneExperimentalVisitor$@264804f5),
(JoinSelectivityOneGuard,com.databricks.sql.optimizer.statsEstimation.experiments.J
oinSelectivityOneGuardExperimentalVisitor$@1c46de4f),
(LimitStats,com.databricks.sql.optimizer.statsEstimation.experiments.LimitStatsLogi
calPlanStatsVisitor$@3d069493),
(PropagateStatsJoinedExpressions,com.databricks.sql.optimizer.statsEstimation.exper
iments.PropagateStatsForJoinedExpressionsExperimentalVisitor$@4d9fa535))
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:49.
Current active queries:4
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:48.
Current active queries:5
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 1, New parallelism: 4
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar at
(spark://10.183.232.44:36145/jars/com_databricks_spark_xml_2_12_0_16_0.jar,Some(/
local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:47.
Current active queries:4
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_collection_compat_2_
12_2_9_0.jar,Some(/local_disk0/tmp/
addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar)) has been added
already. Overwriting of added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:46.
Current active queries:3
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:51.
Current active queries:4
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:50.
Current active queries:5
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 1, New parallelism: 4
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_collection_compat_2_
12_2_9_0.jar,Some(/local_disk0/tmp/
addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar)) has been added
already. Overwriting of added jar is not supported in the current version.
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_ooxml_schemas_4_1_2.jar,Some(/
local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:49.
Current active queries:4
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:48.
Current active queries:3
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_ooxml_schemas_4_1_2.jar,Some(/
local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:52.
Current active queries:4
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar at
(spark://10.183.232.44:36145/jars/com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar,Some(/
local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadAvgHelper: Current cluster load: 4, Old Ema: 1.0,
New Ema: 1.45
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:53.
Current active queries:5
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 1, New parallelism: 4
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:51.
Current active queries:4
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:50.
Current active queries:3
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:54.
Current active queries:4
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 1, New parallelism: 5
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_ooxml_schemas_4_1_2.jar,Some(/
local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Added query with execution ID:55.
Current active queries:5
25/02/06 06:23:10 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 1, New parallelism: 4
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:52.
Current active queries:4
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:10 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_lang3_3_9.jar,Some(/
local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:10 INFO ClusterLoadMonitor: Removed query with execution ID:53.
Current active queries:3
25/02/06 06:23:10 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadAvgHelper: Current cluster load: 3, Old Ema:
1.45, New Ema: 1.6824999999999999
25/02/06 06:23:21 INFO ClusterLoadAvgHelper: Current cluster load: 3, Old Ema:
1.6824999999999999, New Ema: 1.8801249999999998
25/02/06 06:23:21 INFO ClusterLoadAvgHelper: Current cluster load: 3, Old Ema:
1.8801249999999998, New Ema: 2.04810625
25/02/06 06:23:21 INFO TaskSetManager: Starting task 2.0 in stage 465.0 (TID 2016)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 3.0 in stage 465.0 (TID 2017)
(10.183.232.45, executor 2, partition 3, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 4.0 in stage 465.0 (TID 2018)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 5.0 in stage 465.0 (TID 2019)
(10.183.232.45, executor 2, partition 5, PROCESS_LOCAL,
25/02/06 06:23:21 ERROR DriverCorral: Could not get repl, python repl for context
237442136387608794 is not ready or in a bad state.
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.229.123:38441 in memory (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_113_piece0 on
10.183.229.123:38441 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:56.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:57.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_lang3_3_9.jar,Some(/
local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.232.44:37705 in memory (size: 126.4 KiB, free: 20.2 GiB)
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar,
Some(/local_disk0/tmp/
addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_lang3_3_9.jar,Some(/
local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar at
(spark://10.183.232.44:36145/jars/org_glassfish_jaxb_txw2_3_0_2.jar,Some(/
local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 ERROR DriverCorral: Invalid executionContextId:
MessagePacket(jupyter_message,Metadata(None,Some(kernel_info_request),Some(87978529
3777268),Some(1019-090756-
ocity0wb),Some(68060094776458688),None,Some(CommandContext(...)),None),MessagePaylo
ad(Map(),Some(control),Some(Header(Some(kernel_info_request),Some(d3f1aac3-9afb-
4806-a75c-5d2eaaec3810),Some(databricksnbid:879785293777268),Some(),Some(2025-02-
06T06:23:14.746Z),Some(5.4))),None,Map(),List()),null)
25/02/06 06:23:21 WARN DriverDaemon: Unexpected exception:
java.lang.NullPointerException
java.lang.NullPointerException
at
com.databricks.backend.daemon.driver.DriverWrapper.inspectRequest(DriverWrapper.sca
la:397)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$handleRPCRequest$3(DriverCorral.scala:1067)
at scala.Option.map(Option.scala:230)
at
com.databricks.backend.daemon.driver.DriverCorral.com$databricks$backend$daemon$dri
ver$DriverCorral$$handleRPCRequest(DriverCorral.scala:1067)
at com.databricks.backend.daemon.driver.DriverCorral$
$anonfun$receive$1.applyOrElse(DriverCorral.scala:1197)
at com.databricks.backend.daemon.driver.DriverCorral$
$anonfun$receive$1.applyOrElse(DriverCorral.scala:1195)
at com.databricks.rpc.ServerBackend.
$anonfun$internalReceive0$2(ServerBackend.scala:174)
at com.databricks.rpc.ServerBackend$
$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:200)
at com.databricks.rpc.ServerBackend$
$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:200)
at com.databricks.rpc.ServerBackend.internalReceive0(ServerBackend.scala:171)
at com.databricks.rpc.ServerBackend.
$anonfun$internalReceive$1(ServerBackend.scala:147)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.rpc.ServerBackend.withAttributionContext(ServerBackend.scala:22)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at
com.databricks.rpc.ServerBackend.withAttributionTags(ServerBackend.scala:22)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at
com.databricks.rpc.ServerBackend.recordOperationWithResultTags(ServerBackend.scala:
22)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.rpc.ServerBackend.recordOperation(ServerBackend.scala:22)
at com.databricks.rpc.ServerBackend.internalReceive(ServerBackend.scala:147)
at
com.databricks.rpc.JettyServer$RequestManager.handleRPC(JettyServer.scala:1037)
at
com.databricks.rpc.JettyServer$RequestManager.handleRequestAndRespond(JettyServer.s
cala:948)
at com.databricks.rpc.JettyServer$RequestManager.
$anonfun$handleHttp$6(JettyServer.scala:540)
at com.databricks.rpc.JettyServer$RequestManager.
$anonfun$handleHttp$6$adapted(JettyServer.scala:515)
at com.databricks.logging.activity.ActivityContextFactory$.
$anonfun$withActivityInternal$3(ActivityContextFactory.scala:420)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.logging.activity.ActivityContextFactory$.withAttributionContext(Acti
vityContextFactory.scala:55)
at
com.databricks.logging.activity.ActivityContextFactory$.withActivityInternal(Activi
tyContextFactory.scala:420)
at
com.databricks.logging.activity.ActivityContextFactory$.withServiceRequestActivity(
ActivityContextFactory.scala:179)
at
com.databricks.rpc.JettyServer$RequestManager.handleHttp(JettyServer.scala:515)
at
com.databricks.rpc.JettyServer$RequestManager.doPost(JettyServer.scala:404)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
at
com.databricks.rpc.HttpServletWithPatch.service(HttpServletWithPatch.scala:33)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.j
ava:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:3
38)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java
:315)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.jav
a:173)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThr
eadExecutor.java:409)
at com.databricks.rpc.InstrumentedQueuedThreadPool$
$anon$1.$anonfun$run$1(InstrumentedQueuedThreadPool.scala:83)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.instrumentation.QueuedThreadPoolInstrumenter.trackActiveThreads(Queu
edThreadPoolInstrumenter.scala:66)
at
com.databricks.instrumentation.QueuedThreadPoolInstrumenter.trackActiveThreads$
(QueuedThreadPoolInstrumenter.scala:63)
at
com.databricks.rpc.InstrumentedQueuedThreadPool.trackActiveThreads(InstrumentedQueu
edThreadPool.scala:49)
at com.databricks.rpc.InstrumentedQueuedThreadPool$
$anon$1.run(InstrumentedQueuedThreadPool.scala:78)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:103
4)
at java.lang.Thread.run(Thread.java:750)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 8.0 in stage 466.0 (TID 2004)
in 11597 ms on 10.183.232.45 (executor 2) (1/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 6.0 in stage 466.0 (TID 2002)
in 11597 ms on 10.183.232.45 (executor 2) (2/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 9.0 in stage 466.0 (TID 2005)
in 11596 ms on 10.183.232.45 (executor 2) (3/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 7.0 in stage 466.0 (TID 2003)
in 11597 ms on 10.183.232.45 (executor 2) (4/18)
25/02/06 06:23:21 INFO TaskSetManager: Starting task 6.0 in stage 465.0 (TID 2020)
(10.183.249.197, executor 7, partition 6, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 7.0 in stage 465.0 (TID 2021)
(10.183.249.197, executor 7, partition 7, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 8.0 in stage 465.0 (TID 2022)
(10.183.249.197, executor 7, partition 8, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 9.0 in stage 465.0 (TID 2023)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 10.0 in stage 465.0 (TID 2024)
(10.183.229.123, executor 0, partition 10, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 11.0 in stage 465.0 (TID 2025)
(10.183.229.123, executor 0, partition 11, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 12.0 in stage 465.0 (TID 2026)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 13.0 in stage 465.0 (TID 2027)
(10.183.229.123, executor 0, partition 13, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 14.0 in stage 465.0 (TID 2028)
(10.183.233.191, executor 5, partition 14, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 15.0 in stage 465.0 (TID 2029)
(10.183.233.191, executor 5, partition 15, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 16.0 in stage 465.0 (TID 2030)
(10.183.233.191, executor 5, partition 16, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 17.0 in stage 465.0 (TID 2031)
(10.183.233.191, executor 5, partition 17, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 0.0 in stage 467.0 (TID 2032)
(10.183.246.72, executor 6, partition 0, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 1.0 in stage 467.0 (TID 2033)
(10.183.246.72, executor 6, partition 1, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 2.0 in stage 467.0 (TID 2034)
(10.183.246.72, executor 6, partition 2, PROCESS_LOCAL,
25/02/06 06:23:21 INFO TaskSetManager: Starting task 3.0 in stage 467.0 (TID 2035)
(10.183.246.72, executor 6, partition 3, PROCESS_LOCAL,
25/02/06 06:23:21 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on
10.183.232.45:45579 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_113_piece0 on
10.183.246.72:43231 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.246.72:43231 in memory (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_113_piece0 on
10.183.232.44:37705 in memory (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 0.0 in stage 465.0 (TID 2014)
in 11623 ms on 10.183.249.197 (executor 7) (1/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 17.0 in stage 466.0 (TID 2013)
in 11629 ms on 10.183.249.197 (executor 7) (5/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 10.0 in stage 466.0 (TID 2006)
in 11648 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 15.0 in stage 466.0 (TID 2011)
in 11630 ms on 10.183.249.197 (executor 7) (7/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 13.0 in stage 466.0 (TID 2009)
in 11649 ms on 10.183.233.191 (executor 5) (8/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 1.0 in stage 465.0 (TID 2015)
in 11631 ms on 10.183.249.197 (executor 7) (2/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 12.0 in stage 466.0 (TID 2008)
in 11649 ms on 10.183.229.123 (executor 0) (9/18)
25/02/06 06:23:21 INFO MemoryStore: Block broadcast_117 stored as values in memory
(estimated size 349.8 KiB, free 20.2 GiB)
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:55.
Current active queries:1
25/02/06 06:23:21 INFO TaskSetManager: Finished task 5.0 in stage 466.0 (TID 2001)
in 22122 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:57.
Current active queries:2
25/02/06 06:23:21 INFO TaskSetManager: Finished task 2.0 in stage 466.0 (TID 1998)
in 22122 ms on 10.183.246.72 (executor 6) (11/18)
25/02/06 06:23:21 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:56.
Current active queries:3
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:54.
Current active queries:4
25/02/06 06:23:21 INFO TaskSetManager: Finished task 4.0 in stage 466.0 (TID 2000)
in 22123 ms on 10.183.229.123 (executor 0) (12/18)
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO TaskSetManager: Finished task 3.0 in stage 466.0 (TID 1999)
in 22124 ms on 10.183.246.72 (executor 6) (13/18)
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has
been deprecated in Spark v3.1 and may be removed in the future. Set
'spark.sql.legacy.createHiveTableByDefault' to false instead.
25/02/06 06:23:21 INFO TaskSetManager: Finished task 1.0 in stage 466.0 (TID 1997)
in 22126 ms on 10.183.246.72 (executor 6) (14/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 16.0 in stage 466.0 (TID 2012)
in 11636 ms on 10.183.233.191 (executor 5) (15/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 0.0 in stage 466.0 (TID 1996)
in 22130 ms on 10.183.246.72 (executor 6) (16/18)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 11.0 in stage 466.0 (TID 2007)
in 11657 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:23:21 INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:23:21 INFO TaskSetManager: Finished task 14.0 in stage 466.0 (TID 2010)
in 11654 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:23:21 INFO TaskSchedulerImpl: Removed TaskSet 466.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:23:21 INFO DAGScheduler: ShuffleMapStage 466
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
251.083 s
25/02/06 06:23:21 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:23:21 INFO DAGScheduler: running: Set(ShuffleMapStage 481,
ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577, ShuffleMapStage 556,
ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 602,
ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517,
ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490,
ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 646, ShuffleMapStage 465, ShuffleMapStage 559, ShuffleMapStage 538,
ShuffleMapStage 532, ShuffleMapStage 511, ShuffleMapStage 613, ShuffleMapStage 505,
ShuffleMapStage 484, ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580,
ShuffleMapStage 624, ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568,
ShuffleMapStage 547, ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591,
ShuffleMapStage 541, ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 514,
ShuffleMapStage 493, ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:23:21 INFO DAGScheduler: waiting: Set()
25/02/06 06:23:21 INFO DAGScheduler: failed: Set()
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:59.
Current active queries:2
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:61.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:60.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:58.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.233.191:42923 in memory (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_113_piece0 on
10.183.232.45:45579 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.232.45:45579 in memory (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_114_piece0 on
10.183.249.197:40023 in memory (size: 126.4 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Removed broadcast_113_piece0 on
10.183.249.197:40023 in memory (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_core_4_8_2.jar,Some(/
local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:61.
Current active queries:4
25/02/06 06:23:21 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on
10.183.229.123:38441 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_math3_3_6_1.jar,Some(/
local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:59.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:62.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on
10.183.233.191:42923 (size: 126.2 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:63.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar at
(spark://10.183.232.44:36145/jars/org_glassfish_jaxb_txw2_3_0_2.jar,Some(/
local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO SparkContext: Created broadcast 117 from broadcast at
TaskSetManager.scala:711
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar at
(spark://10.183.232.44:36145/jars/org_glassfish_jaxb_txw2_3_0_2.jar,Some(/
local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.246.72:43231 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:58.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:60.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:64.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:65.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_core_4_8_2.jar,Some(/
local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_record_codec_4_8_2.jar,Some(/
local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:62.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:65.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:66.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_core_4_8_2.jar,Some(/
local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:64.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_collection_compat_2_
12_2_9_0.jar,Some(/local_disk0/tmp/
addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar)) has been added
already. Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:63.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:67.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:68.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_sync_4_8_2.jar,Some(/
local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:69.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:66.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:70.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO DriverCorral: Starting python repl ReplId-70d0c-4a4fd-a2328-
a
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_record_codec_4_8_2.jar,Some(/
local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_record_codec_4_8_2.jar,Some(/
local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_ooxml_schemas_4_1_2.jar,Some(/
local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:67.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:69.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:68.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:71.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:73.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar at
(spark://10.183.232.44:36145/jars/org_slf4j_slf4j_api_1_7_30.jar,Some(/
local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:70.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:72.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_sync_4_8_2.jar,Some(/
local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:71.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:74.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_lang3_3_9.jar,Some(/
local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:72.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:75.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:21 INFO BaselineLogicalPlanStats: Setting BaselineLogicalPlanStats
visitor to
com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ExperimentalLogicalPlanStats: Setting
ExperimentalLogicalPlanStats visitors to
List((NonEquiJoinCondition,com.databricks.sql.optimizer.statsEstimation.experiments
.NonEquiJoinConditionLogicalPlanStatsVisitor$@33a2586c),
(WindowStats,com.databricks.sql.optimizer.statsEstimation.experiments.WindowStatsLo
gicalPlanStatsVisitor$@4cd787c9),
(JoinSelectivityOne,com.databricks.sql.optimizer.statsEstimation.experiments.JoinSe
lectivityOneExperimentalVisitor$@264804f5),
(JoinSelectivityOneGuard,com.databricks.sql.optimizer.statsEstimation.experiments.J
oinSelectivityOneGuardExperimentalVisitor$@1c46de4f),
(LimitStats,com.databricks.sql.optimizer.statsEstimation.experiments.LimitStatsLogi
calPlanStatsVisitor$@3d069493),
(PropagateStatsJoinedExpressions,com.databricks.sql.optimizer.statsEstimation.exper
iments.PropagateStatsForJoinedExpressionsExperimentalVisitor$@4d9fa535))
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO DriverCorral$: Cleaning the wrapper ReplId-70d0c-4a4fd-
a2328-a (currently in status Starting(ReplId-70d0c-4a4fd-a2328-a))
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:76.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO DriverCorral$: sending shutdown signal for REPL ReplId-
70d0c-4a4fd-a2328-a
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:77.
Current active queries:6
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar at
(spark://10.183.232.44:36145/jars/com_github_virtuald_curvesapi_1_06.jar,Some(/
local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:74.
Current active queries:5
25/02/06 06:23:21 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-
70d0c-4a4fd-a2328-a
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO DriverCorral$: waiting for localThread to stop for REPL
ReplId-70d0c-4a4fd-a2328-a
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_sync_4_8_2.jar,Some(/
local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:73.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar at
(spark://10.183.232.44:36145/jars/org_glassfish_jaxb_txw2_3_0_2.jar,Some(/
local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:76.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar at
(spark://10.183.232.44:36145/jars/com_databricks_spark_xml_2_12_0_16_0.jar,Some(/
local_disk0/tmp/addedFile3c98001de8ca48538fda362c7a389b151738291073093716681/
com_databricks_spark_xml_2_12_0_16_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:75.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:80.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:78.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:79.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_core_4_8_2.jar,Some(/
local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:79.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:82.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:81.
Current active queries:6
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar at
(spark://10.183.232.44:36145/jars/org_slf4j_slf4j_api_1_7_30.jar,Some(/
local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_poi_shared_strings_1_0_4.jar
,Some(/local_disk0/tmp/
addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar at
(spark://10.183.232.44:36145/jars/com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar,Some(/
local_disk0/tmp/addedFile17c7819a0f0c4f7fa7f0a61e959888705066313782367831337/
com_sap_cloud_db_jdbc_ngdbc_2_19_16.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar at
(spark://10.183.232.44:36145/jars/org_slf4j_slf4j_api_1_7_30.jar,Some(/
local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:77.
Current active queries:5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:80.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:82.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:78.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:84.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:85.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:86.
Current active queries:5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar,
Some(/local_disk0/tmp/
addedFile876e65c54501480d98e2df13fa5046c45749853013669560262/
org_apache_ws_xmlschema_xmlschema_core_2_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:84.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:83.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar at
(spark://10.183.232.44:36145/jars/com_github_virtuald_curvesapi_1_06.jar,Some(/
local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_record_codec_4_8_2.jar,Some(/
local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:81.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:85.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:87.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar at
(spark://10.183.232.44:36145/jars/com_github_virtuald_curvesapi_1_06.jar,Some(/
local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:83.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar at
(spark://10.183.232.44:36145/jars/com_norbitltd_spoiwo_2_12_1_8_0.jar,Some(/
local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:86.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:88.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_poi_shared_strings_1_0_4.jar
,Some(/local_disk0/tmp/
addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_sync_4_8_2.jar,Some(/
local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:88.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:87.
Current active queries:1
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:89.
Current active queries:2
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:92.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:91.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_poi_shared_strings_1_0_4.jar
,Some(/local_disk0/tmp/
addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:92.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_compress_1_21.jar,Some
(/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar)) has been added already. Overwriting
of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_math3_3_6_1.jar,Some(/
local_disk0/tmp/addedFile88b2ed8e3eaa46c792ef86bee6344ec56077973845506247562/
org_apache_commons_commons_math3_3_6_1.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:90.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:89.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:91.
Current active queries:2
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:93.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:94.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:95.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar at
(spark://10.183.232.44:36145/jars/com_norbitltd_spoiwo_2_12_1_8_0.jar,Some(/
local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:90.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar at
(spark://10.183.232.44:36145/jars/com_norbitltd_spoiwo_2_12_1_8_0.jar,Some(/
local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:94.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar at
(spark://10.183.232.44:36145/jars/org_slf4j_slf4j_api_1_7_30.jar,Some(/
local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:93.
Current active queries:2
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:97.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_collections4_4_4.jar,S
ome(/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_collection_compat_2_
12_2_9_0.jar,Some(/local_disk0/tmp/
addedFile7afac28405b342dbaf20d4d5913db1182275993060596004368/
org_scala_lang_modules_scala_collection_compat_2_12_2_9_0.jar)) has been added
already. Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:97.
Current active queries:2
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:95.
Current active queries:1
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:99.
Current active queries:2
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:98.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:96.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:100.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:101.
Current active queries:6
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_compress_1_21.jar,Some
(/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar)) has been added already. Overwriting
of added jar is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar at
(spark://10.183.232.44:36145/jars/com_github_virtuald_curvesapi_1_06.jar,Some(/
local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:99.
Current active queries:5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:98.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_compress_1_21.jar,Some
(/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar)) has been added already. Overwriting
of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:96.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:102.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:103.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_ooxml_schemas_4_1_2.jar,Some(/
local_disk0/tmp/addedFilea191a31e4ec54a9dadc416bbf6b1a8c86524044965486518662/
org_apache_poi_poi_ooxml_schemas_4_1_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:100.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_4_1_2.jar,Some(/local_disk0/
tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:101.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_collections4_4_4.jar,S
ome(/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:104.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:102.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:105.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:106.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_collections4_4_4.jar,S
ome(/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:103.
Current active queries:4
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:107.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_poi_shared_strings_1_0_4.jar
,Some(/local_disk0/tmp/
addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:104.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar at
(spark://10.183.232.44:36145/jars/com_zaxxer_SparseBitSet_1_2.jar,Some(/
local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:106.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_lang3_3_9.jar,Some(/
local_disk0/tmp/addedFilecf7fe642b5b041bca5d826637f7f7af06660653383349414969/
org_apache_commons_commons_lang3_3_9.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:105.
Current active queries:2
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:109.
Current active queries:3
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:108.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_4_1_2.jar,Some(/local_disk0/
tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:107.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:110.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:111.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_spark_mongo_spark_connector_2_12_10_1
_1.jar,Some(/local_disk0/tmp/
addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:110.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_4_1_2.jar,Some(/local_disk0/
tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:112.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:108.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar at
(spark://10.183.232.44:36145/jars/com_norbitltd_spoiwo_2_12_1_8_0.jar,Some(/
local_disk0/tmp/addedFile3b0c9258f0a34a038472f6c96104f1bc5520783764573936653/
com_norbitltd_spoiwo_2_12_1_8_0.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:109.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:113.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:114.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar at
(spark://10.183.232.44:36145/jars/org_glassfish_jaxb_txw2_3_0_2.jar,Some(/
local_disk0/tmp/addedFile6c862ff1c41c491d95ed5b9a64de15993372116380026620408/
org_glassfish_jaxb_txw2_3_0_2.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:111.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 ERROR DriverCorral: Could not get repl, python repl for context
237442136387608794 is not ready or in a bad state.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar at
(spark://10.183.232.44:36145/jars/commons_codec_commons_codec_1_13.jar,Some(/
local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:113.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar at
(spark://10.183.232.44:36145/jars/com_zaxxer_SparseBitSet_1_2.jar,Some(/
local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:115.
Current active queries:4
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:116.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:112.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:117.
Current active queries:5
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Added query with execution ID:118.
Current active queries:6
25/02/06 06:23:21 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar at
(spark://10.183.232.44:36145/jars/com_zaxxer_SparseBitSet_1_2.jar,Some(/
local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_excel_streaming_reader_2_3_6
.jar,Some(/local_disk0/tmp/
addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:114.
Current active queries:5
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:116.
Current active queries:4
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:21 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_compress_1_21.jar,Some
(/local_disk0/tmp/addedFile78947bbda9154f63801db72d41d88ff61497026711532725734/
org_apache_commons_commons_compress_1_21.jar)) has been added already. Overwriting
of added jar is not supported in the current version.
25/02/06 06:23:21 INFO ClusterLoadMonitor: Removed query with execution ID:115.
Current active queries:3
25/02/06 06:23:21 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_spark_mongo_spark_connector_2_12_10_1
_1.jar,Some(/local_disk0/tmp/
addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:118.
Current active queries:2
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 ERROR DriverCorral: Could not get repl, python repl for context
237442136387608794 is not ready or in a bad state.
25/02/06 06:23:33 INFO TaskSetManager: Finished task 1.0 in stage 467.0 (TID 2033)
in 11852 ms on 10.183.246.72 (executor 6) (1/18)
25/02/06 06:23:33 INFO TaskSetManager: Starting task 4.0 in stage 467.0 (TID 2036)
(10.183.246.72, executor 6, partition 4, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Finished task 0.0 in stage 467.0 (TID 2032)
in 11854 ms on 10.183.246.72 (executor 6) (2/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 3.0 in stage 467.0 (TID 2035)
in 11853 ms on 10.183.246.72 (executor 6) (3/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 2.0 in stage 467.0 (TID 2034)
in 11854 ms on 10.183.246.72 (executor 6) (4/18)
25/02/06 06:23:33 INFO TaskSetManager: Starting task 5.0 in stage 467.0 (TID 2037)
(10.183.246.72, executor 6, partition 5, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 6.0 in stage 467.0 (TID 2038)
(10.183.246.72, executor 6, partition 6, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 7.0 in stage 467.0 (TID 2039)
(10.183.246.72, executor 6, partition 7, PROCESS_LOCAL,
25/02/06 06:23:33 INFO ClusterLoadAvgHelper: Current cluster load: 2, Old Ema:
2.04810625, New Ema: 2.0408903124999997
25/02/06 06:23:33 INFO ClusterLoadAvgHelper: Current cluster load: 2, Old Ema:
2.0408903124999997, New Ema: 2.0347567656249996
25/02/06 06:23:33 INFO ClusterLoadAvgHelper: Current cluster load: 2, Old Ema:
2.0347567656249996, New Ema: 2.0295432507812494
25/02/06 06:23:33 INFO ClusterLoadAvgHelper: Current cluster load: 2, Old Ema:
2.0295432507812494, New Ema: 2.025111763164062
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.233.191:42923 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_core_4_8_2.jar,Some(/
local_disk0/tmp/addedFilee46ed771a73c4bb8b28d147bfed711f94545930638235211975/
org_mongodb_mongodb_driver_core_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:117.
Current active queries:1
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.229.123:38441 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN DynamicSparkConfContextImpl: Ignored update because id
1738821253463 < 1738821253463; source: ConfigFile
25/02/06 06:23:33 INFO DriverCorral: Received SAFEr configs with version
1738821253463
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:121.
Current active queries:2
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.249.197:40023 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.232.45:45579 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 10.0 in stage 465.0 (TID 2024)
in 11886 ms on 10.183.229.123 (executor 0) (3/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 14.0 in stage 465.0 (TID 2028)
in 11887 ms on 10.183.233.191 (executor 5) (4/18)
25/02/06 06:23:33 INFO TaskSetManager: Starting task 8.0 in stage 467.0 (TID 2040)
(10.183.229.123, executor 0, partition 8, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 9.0 in stage 467.0 (TID 2041)
(10.183.249.197, executor 7, partition 9, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 10.0 in stage 467.0 (TID 2042)
(10.183.232.45, executor 2, partition 10, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 11.0 in stage 467.0 (TID 2043)
(10.183.233.191, executor 5, partition 11, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 12.0 in stage 467.0 (TID 2044)
(10.183.229.123, executor 0, partition 12, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 13.0 in stage 467.0 (TID 2045)
(10.183.249.197, executor 7, partition 13, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 14.0 in stage 467.0 (TID 2046)
(10.183.232.45, executor 2, partition 14, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 15.0 in stage 467.0 (TID 2047)
(10.183.233.191, executor 5, partition 15, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 16.0 in stage 467.0 (TID 2048)
(10.183.229.123, executor 0, partition 16, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 17.0 in stage 467.0 (TID 2049)
(10.183.249.197, executor 7, partition 17, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 0.0 in stage 468.0 (TID 2050)
(10.183.229.123, executor 0, partition 0, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 1.0 in stage 468.0 (TID 2051)
(10.183.249.197, executor 7, partition 1, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 2.0 in stage 468.0 (TID 2052)
(10.183.232.45, executor 2, partition 2, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 3.0 in stage 468.0 (TID 2053)
(10.183.233.191, executor 5, partition 3, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 4.0 in stage 468.0 (TID 2054)
(10.183.232.45, executor 2, partition 4, PROCESS_LOCAL,
25/02/06 06:23:33 INFO TaskSetManager: Starting task 5.0 in stage 468.0 (TID 2055)
(10.183.233.191, executor 5, partition 5, PROCESS_LOCAL,
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:119.
Current active queries:3
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:33 WARN DriverCorral$: REPL ReplId-70d0c-4a4fd-a2328-a did not stop
after interrupt; localThread stack trace is:
java.lang.Throwable.getStackTraceElement(Native Method)
java.lang.Throwable.getOurStackTrace(Throwable.java:828)
java.lang.Throwable.getStackTrace(Throwable.java:817)
java.lang.Thread.getStackTrace(Thread.java:1564)
org.apache.spark.util.Utils$.getCallSite(Utils.scala:1878)
org.apache.spark.SparkContext.callSite$lzycompute$1(SparkContext.scala:2885)
org.apache.spark.SparkContext.callSite$2(SparkContext.scala:2885)
org.apache.spark.SparkContext.$anonfun$getCallSite$1(SparkContext.scala:2887)
org.apache.spark.SparkContext$$Lambda$5212/1665175449.apply(Unknown Source)
scala.Option.getOrElse(Option.scala:189)
org.apache.spark.SparkContext.getCallSite(SparkContext.scala:2887)
org.apache.spark.sql.execution.SQLExecution$.
$anonfun$withCustomExecutionEnv$1(SQLExecution.scala:203)
org.apache.spark.sql.execution.SQLExecution$$$Lambda$5210/1584310698.apply(Unknown
Source)
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1148)
org.apache.spark.sql.execution.SQLExecution$.withCustomExecutionEnv(SQLExecution.sc
ala:155)
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:
482)
org.apache.spark.sql.execution.QueryExecution$$anonfun$
$nestedInanonfun$eagerlyExecuteCommands$1$1.$anonfun$applyOrElse$1(QueryExecution.s
cala:285)
org.apache.spark.sql.execution.QueryExecution$$anonfun$
$nestedInanonfun$eagerlyExecuteCommands$1$1$$Lambda$5208/1086953428.apply(Unknown
Source)
org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryE
xecution$$withMVTagsIfNecessary(QueryExecution.scala:259)
org.apache.spark.sql.execution.QueryExecution$$anonfun$
$nestedInanonfun$eagerlyExecuteCommands$1$1.applyOrElse(QueryExecution.scala:280)
org.apache.spark.sql.execution.QueryExecution$$anonfun$
$nestedInanonfun$eagerlyExecuteCommands$1$1.applyOrElse(QueryExecution.scala:265)
org.apache.spark.sql.catalyst.trees.TreeNode.
$anonfun$transformDownWithPruning$1(TreeNode.scala:465)
org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$3651/1866191073.apply(Unknown
Source)
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:69)
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scal
a:465)
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$cataly
st$plans$logical$AnalysisHelper$
$super$transformDownWithPruning(LogicalPlan.scala:39)
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning
(AnalysisHelper.scala:339)
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning
$(AnalysisHelper.scala:335)
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(Lo
gicalPlan.scala:39)
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(Lo
gicalPlan.scala:39)
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:441)
org.apache.spark.sql.execution.QueryExecution.
$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:265)
org.apache.spark.sql.execution.QueryExecution$$Lambda$3875/1290654769.apply(Unknown
Source)
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransforms
InAnalyzer(AnalysisHelper.scala:395)
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution
.scala:265)
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecu
tion.scala:217)
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:
214)
org.apache.spark.sql.Dataset.<init>(Dataset.scala:261)
org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:122)
org.apache.spark.sql.Dataset$$$Lambda$3501/1827943003.apply(Unknown Source)
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1148)
org.apache.spark.sql.SparkSession.
$anonfun$withActiveAndFrameProfiler$1(SparkSession.scala:1155)
org.apache.spark.sql.SparkSession$$Lambda$3502/795566318.apply(Unknown Source)
com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
org.apache.spark.sql.SparkSession.withActiveAndFrameProfiler(SparkSession.scala:115
5)
org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:112)
org.apache.spark.sql.SparkSession.$anonfun$sql$5(SparkSession.scala:928)
org.apache.spark.sql.SparkSession$$Lambda$3166/225769102.apply(Unknown Source)
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1148)
org.apache.spark.sql.SparkSession.sql(SparkSession.scala:917)
org.apache.spark.sql.SparkSession.$anonfun$sql$9(SparkSession.scala:951)
org.apache.spark.sql.SparkSession$$Lambda$3165/1563640426.apply(Unknown Source)
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1148)
org.apache.spark.sql.SparkSession.sql(SparkSession.scala:951)
org.apache.spark.sql.SparkSession.sql(SparkSession.scala:984)
org.apache.spark.sql.SQLContext.sql(SQLContext.scala:695)
com.databricks.backend.daemon.driver.DriverLocal.
$anonfun$new$6(DriverLocal.scala:497)
com.databricks.backend.daemon.driver.DriverLocal$
$Lambda$5202/1232933154.apply(Unknown Source)
org.apache.spark.SafeAddJarOrFile$.safe(SafeAddJarOrFile.scala:31)
com.databricks.backend.daemon.driver.DriverLocal.
$anonfun$new$5(DriverLocal.scala:497)
com.databricks.backend.daemon.driver.DriverLocal$
$Lambda$5200/2083214544.apply(Unknown Source)
com.databricks.sql.acl.CheckPermissions$.
$anonfun$trusted$1(CheckPermissions.scala:2037)
com.databricks.sql.acl.CheckPermissions$$$Lambda$5201/2072300272.apply(Unknown
Source)
com.databricks.sql.util.ThreadLocalTagger.withTag(QueryTagger.scala:62)
com.databricks.sql.util.ThreadLocalTagger.withTag$(QueryTagger.scala:59)
com.databricks.sql.util.QueryTagger$.withTag(QueryTagger.scala:130)
com.databricks.sql.acl.CheckPermissions$.trusted(CheckPermissions.scala:2037)
com.databricks.backend.daemon.driver.DriverLocal.
$anonfun$new$4(DriverLocal.scala:496)
com.databricks.backend.daemon.driver.DriverLocal$
$Lambda$5199/1121177300.apply(Unknown Source)
com.databricks.unity.UCSEphemeralState$Handle.runWith(UCSEphemeralState.scala:45)
com.databricks.unity.HandleImpl.runWith(UCSHandle.scala:103)
com.databricks.backend.daemon.driver.DriverLocal.
$anonfun$new$3(DriverLocal.scala:489)
com.databricks.backend.daemon.driver.DriverLocal$
$Lambda$5198/249955338.apply(Unknown Source)
scala.util.Using$.resource(Using.scala:269)
com.databricks.backend.daemon.driver.DriverLocal.
$anonfun$new$2(DriverLocal.scala:488)
com.databricks.backend.daemon.driver.DriverLocal$
$Lambda$4781/917963793.apply(Unknown Source)
scala.collection.Iterator.foreach(Iterator.scala:943)
scala.collection.Iterator.foreach$(Iterator.scala:943)
scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
scala.collection.IterableLike.foreach(IterableLike.scala:74)
scala.collection.IterableLike.foreach$(IterableLike.scala:73)
scala.collection.AbstractIterable.foreach(Iterable.scala:56)
com.databricks.backend.daemon.driver.DriverLocal.<init>(DriverLocal.scala:475)
com.databricks.backend.daemon.driver.PythonDriverLocalBase.<init>(PythonDriverLocal
Base.scala:191)
com.databricks.backend.daemon.driver.JupyterDriverLocal.<init>(JupyterDriverLocal.s
cala:195)
com.databricks.backend.daemon.driver.PythonDriverWrapper.instantiateDriver(DriverWr
apper.scala:1038)
com.databricks.backend.daemon.driver.DriverWrapper.setupRepl(DriverWrapper.scala:41
4)
com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:295)
java.lang.Thread.run(Thread.java:750)
25/02/06 06:23:33 INFO MemoryStore: Block broadcast_118 stored as values in memory
(estimated size 349.7 KiB, free 20.2 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.232.44:37705 in memory (size: 126.0 KiB, free: 20.2 GiB)
25/02/06 06:23:33 INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in
memory (estimated size 126.1 KiB, free 20.2 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on
10.183.232.44:37705 (size: 126.1 KiB, free: 20.2 GiB)
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:120.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:122.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 WARN DriverDaemon: Unexpected exception:
java.lang.NullPointerException
java.lang.NullPointerException
at
com.databricks.backend.daemon.driver.DriverWrapper.inspectRequest(DriverWrapper.sca
la:397)
at com.databricks.backend.daemon.driver.DriverCorral.
$anonfun$handleRPCRequest$3(DriverCorral.scala:1067)
at scala.Option.map(Option.scala:230)
at
com.databricks.backend.daemon.driver.DriverCorral.com$databricks$backend$daemon$dri
ver$DriverCorral$$handleRPCRequest(DriverCorral.scala:1067)
at com.databricks.backend.daemon.driver.DriverCorral$
$anonfun$receive$1.applyOrElse(DriverCorral.scala:1197)
at com.databricks.backend.daemon.driver.DriverCorral$
$anonfun$receive$1.applyOrElse(DriverCorral.scala:1195)
at com.databricks.rpc.ServerBackend.
$anonfun$internalReceive0$2(ServerBackend.scala:174)
at com.databricks.rpc.ServerBackend$
$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:200)
at com.databricks.rpc.ServerBackend$
$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:200)
at com.databricks.rpc.ServerBackend.internalReceive0(ServerBackend.scala:171)
at com.databricks.rpc.ServerBackend.
$anonfun$internalReceive$1(ServerBackend.scala:147)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperation$1(UsageLogging.scala:571)
at
com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging
.scala:667)
at com.databricks.logging.UsageLogging.
$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:685)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.rpc.ServerBackend.withAttributionContext(ServerBackend.scala:22)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:470)
at com.databricks.logging.UsageLogging.withAttributionTags$
(UsageLogging.scala:455)
at
com.databricks.rpc.ServerBackend.withAttributionTags(ServerBackend.scala:22)
at
com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scal
a:662)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$
(UsageLogging.scala:580)
at
com.databricks.rpc.ServerBackend.recordOperationWithResultTags(ServerBackend.scala:
22)
at
com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:571)
at com.databricks.logging.UsageLogging.recordOperation$
(UsageLogging.scala:540)
at com.databricks.rpc.ServerBackend.recordOperation(ServerBackend.scala:22)
at com.databricks.rpc.ServerBackend.internalReceive(ServerBackend.scala:147)
at
com.databricks.rpc.JettyServer$RequestManager.handleRPC(JettyServer.scala:1037)
at
com.databricks.rpc.JettyServer$RequestManager.handleRequestAndRespond(JettyServer.s
cala:948)
at com.databricks.rpc.JettyServer$RequestManager.
$anonfun$handleHttp$6(JettyServer.scala:540)
at com.databricks.rpc.JettyServer$RequestManager.
$anonfun$handleHttp$6$adapted(JettyServer.scala:515)
at com.databricks.logging.activity.ActivityContextFactory$.
$anonfun$withActivityInternal$3(ActivityContextFactory.scala:420)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.logging.activity.ActivityContextFactory$.withAttributionContext(Acti
vityContextFactory.scala:55)
at
com.databricks.logging.activity.ActivityContextFactory$.withActivityInternal(Activi
tyContextFactory.scala:420)
at
com.databricks.logging.activity.ActivityContextFactory$.withServiceRequestActivity(
ActivityContextFactory.scala:179)
at
com.databricks.rpc.JettyServer$RequestManager.handleHttp(JettyServer.scala:515)
at
com.databricks.rpc.JettyServer$RequestManager.doPost(JettyServer.scala:404)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
at
com.databricks.rpc.HttpServletWithPatch.service(HttpServletWithPatch.scala:33)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.j
ava:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:3
38)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java
:315)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.jav
a:173)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThr
eadExecutor.java:409)
at com.databricks.rpc.InstrumentedQueuedThreadPool$
$anon$1.$anonfun$run$1(InstrumentedQueuedThreadPool.scala:83)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.instrumentation.QueuedThreadPoolInstrumenter.trackActiveThreads(Queu
edThreadPoolInstrumenter.scala:66)
at
com.databricks.instrumentation.QueuedThreadPoolInstrumenter.trackActiveThreads$
(QueuedThreadPoolInstrumenter.scala:63)
at
com.databricks.rpc.InstrumentedQueuedThreadPool.trackActiveThreads(InstrumentedQueu
edThreadPool.scala:49)
at com.databricks.rpc.InstrumentedQueuedThreadPool$
$anon$1.run(InstrumentedQueuedThreadPool.scala:78)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:103
4)
at java.lang.Thread.run(Thread.java:750)
25/02/06 06:23:33 INFO BlockManagerInfo: Removed broadcast_115_piece0 on
10.183.246.72:43231 in memory (size: 126.0 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO DriverCorral$: ReplId-70d0c-4a4fd-a2328-a successfully
discarded
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_spark_mongo_spark_connector_2_12_10_1
_1.jar,Some(/local_disk0/tmp/
addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:120.
Current active queries:4
25/02/06 06:23:33 INFO TaskSetManager: Finished task 11.0 in stage 465.0 (TID 2025)
in 11938 ms on 10.183.229.123 (executor 0) (5/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 12.0 in stage 465.0 (TID 2026)
in 11939 ms on 10.183.229.123 (executor 0) (6/18)
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar at
(spark://10.183.232.44:36145/jars/commons_codec_commons_codec_1_13.jar,Some(/
local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:121.
Current active queries:3
25/02/06 06:23:33 INFO TaskSetManager: Finished task 16.0 in stage 465.0 (TID 2030)
in 11939 ms on 10.183.233.191 (executor 5) (7/18)
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO TaskSetManager: Finished task 9.0 in stage 465.0 (TID 2023)
in 11941 ms on 10.183.249.197 (executor 7) (8/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 8.0 in stage 465.0 (TID 2022)
in 11942 ms on 10.183.249.197 (executor 7) (9/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 13.0 in stage 465.0 (TID 2027)
in 11941 ms on 10.183.229.123 (executor 0) (10/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 7.0 in stage 465.0 (TID 2021)
in 11943 ms on 10.183.249.197 (executor 7) (11/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 6.0 in stage 465.0 (TID 2020)
in 11943 ms on 10.183.249.197 (executor 7) (12/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 2.0 in stage 465.0 (TID 2016)
in 12003 ms on 10.183.232.45 (executor 2) (13/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 3.0 in stage 465.0 (TID 2017)
in 12003 ms on 10.183.232.45 (executor 2) (14/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 5.0 in stage 465.0 (TID 2019)
in 12003 ms on 10.183.232.45 (executor 2) (15/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 4.0 in stage 465.0 (TID 2018)
in 12004 ms on 10.183.232.45 (executor 2) (16/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 17.0 in stage 465.0 (TID 2031)
in 11943 ms on 10.183.233.191 (executor 5) (17/18)
25/02/06 06:23:33 INFO TaskSetManager: Finished task 15.0 in stage 465.0 (TID 2029)
in 11943 ms on 10.183.233.191 (executor 5) (18/18)
25/02/06 06:23:33 INFO TaskSchedulerImpl: Removed TaskSet 465.0, whose tasks have
all completed, from pool 5400065285585087590
25/02/06 06:23:33 INFO DAGScheduler: ShuffleMapStage 465
($anonfun$withThreadLocalCaptured$5 at LexicalThreadLocal.scala:63) finished in
262.968 s
25/02/06 06:23:33 INFO DAGScheduler: looking for newly runnable stages
25/02/06 06:23:33 INFO DAGScheduler: running: Set(ShuffleMapStage 481,
ShuffleMapStage 562, ShuffleMapStage 635, ShuffleMapStage 577, ShuffleMapStage 556,
ShuffleMapStage 535, ShuffleMapStage 550, ShuffleMapStage 529, ShuffleMapStage 602,
ShuffleMapStage 471, ShuffleMapStage 523, ShuffleMapStage 502, ShuffleMapStage 517,
ShuffleMapStage 496, ShuffleMapStage 475, ShuffleMapStage 467, ShuffleMapStage 490,
ShuffleMapStage 469, ShuffleMapStage 571, ShuffleMapStage 565, ShuffleMapStage 544,
ShuffleMapStage 646, ShuffleMapStage 559, ShuffleMapStage 538, ShuffleMapStage 532,
ShuffleMapStage 511, ShuffleMapStage 613, ShuffleMapStage 505, ShuffleMapStage 484,
ShuffleMapStage 499, ShuffleMapStage 478, ShuffleMapStage 580, ShuffleMapStage 624,
ShuffleMapStage 574, ShuffleMapStage 553, ShuffleMapStage 568, ShuffleMapStage 547,
ShuffleMapStage 526, ShuffleMapStage 468, ShuffleMapStage 591, ShuffleMapStage 541,
ShuffleMapStage 520, ShuffleMapStage 470, ShuffleMapStage 514, ShuffleMapStage 493,
ShuffleMapStage 508, ShuffleMapStage 487)
25/02/06 06:23:33 INFO DAGScheduler: waiting: Set()
25/02/06 06:23:33 INFO DAGScheduler: failed: Set()
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:123.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_xml_2_12_1_3_0.jar,S
ome(/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:119.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:124.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_collections4_4_4.jar,S
ome(/local_disk0/tmp/addedFiled9e7fd2a934345f29f1a30f00a5fcf932703966794906716607/
org_apache_commons_commons_collections4_4_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:122.
Current active queries:3
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:125.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO SparkContext: Created broadcast 118 from broadcast at
TaskSetManager.scala:711
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_excel_streaming_reader_2_3_6
.jar,Some(/local_disk0/tmp/
addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:124.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:126.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on
10.183.229.123:38441 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on
10.183.233.191:42923 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:127.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:128.
Current active queries:6
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.249.197:40023 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on
10.183.232.45:45579 (size: 126.1 KiB, free: 18.4 GiB)
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar at
(spark://10.183.232.44:36145/jars/xml_apis_xml_apis_1_4_01.jar,Some(/local_disk0/
tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:125.
Current active queries:5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar at
(spark://10.183.232.44:36145/jars/commons_codec_commons_codec_1_13.jar,Some(/
local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:127.
Current active queries:4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_record_codec_4_8_2.jar,Some(/
local_disk0/tmp/addedFile33e76a196c6d4442b093ccdb5fbf10ff4798143865402977240/
org_mongodb_bson_record_codec_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:123.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:129.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:130.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:131.
Current active queries:6
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_xml_2_12_1_3_0.jar,S
ome(/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:126.
Current active queries:5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_excel_streaming_reader_2_3_6
.jar,Some(/local_disk0/tmp/
addedFiled54b68cf498440eb959801890667382e6014758361423432389/
com_github_pjfanning_excel_streaming_reader_2_3_6.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:129.
Current active queries:4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar at
(spark://10.183.232.44:36145/jars/org_apache_poi_poi_4_1_2.jar,Some(/local_disk0/
tmp/addedFile58c4a8b09ae647af93febc90d546e3473989154229011951782/
org_apache_poi_poi_4_1_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:128.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:132.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_xmlbeans_xmlbeans_3_1_0.jar,Some(/
local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:130.
Current active queries:3
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:133.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:134.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:135.
Current active queries:6
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_mongodb_driver_sync_4_8_2.jar,Some(/
local_disk0/tmp/addedFile8f3b602c815d47e784fa5eaf61b86f585848085492000124591/
org_mongodb_mongodb_driver_sync_4_8_2.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:131.
Current active queries:5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar at
(spark://10.183.232.44:36145/jars/com_zaxxer_SparseBitSet_1_2.jar,Some(/
local_disk0/tmp/addedFile6969028f205440139332d6cff47f3cbc2646382150202389183/
com_zaxxer_SparseBitSet_1_2.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:134.
Current active queries:4
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_text_1_8.jar,Some(/
local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar at
(spark://10.183.232.44:36145/jars/org_scala_lang_modules_scala_xml_2_12_1_3_0.jar,S
ome(/local_disk0/tmp/addedFile89845577b80748b98cc3fa84ce1a63ec7661185990619369740/
org_scala_lang_modules_scala_xml_2_12_1_3_0.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:133.
Current active queries:3
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar at
(spark://10.183.232.44:36145/jars/xml_apis_xml_apis_1_4_01.jar,Some(/local_disk0/
tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:132.
Current active queries:2
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:135.
Current active queries:1
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:139.
Current active queries:2
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:138.
Current active queries:3
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:136.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar at
(spark://10.183.232.44:36145/jars/org_slf4j_slf4j_api_1_7_30.jar,Some(/
local_disk0/tmp/addedFilef38b9325f2664f0da2de736898c0e35e5868366258699122482/
org_slf4j_slf4j_api_1_7_30.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:139.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:137.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:140.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_xmlbeans_xmlbeans_3_1_0.jar,Some(/
local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:136.
Current active queries:4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile0e0d32bfdd08429db5dfcdedd2eef8898265980213250260345/
org_mongodb_bson_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_4_8_2.jar,Some(/local_disk0/
tmp/addedFile0e0d32bfdd08429db5dfcdedd2eef8898265980213250260345/
org_mongodb_bson_4_8_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683297)
25/02/06 06:23:33 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 682561)
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 685385)
25/02/06 06:23:33 WARN BlockManager: Asked to remove block join_relation_683297,
which does not exist
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 684409)
25/02/06 06:23:33 WARN DBCEventLoggingListener: Error in writing the event to log
com.fasterxml.jackson.databind.JsonMappingException: Exceeded 2097152 bytes
(current = 2098441) (through reference chain:
org.apache.spark.sql.execution.ui.SparkListenerSQLAdaptiveExecutionUpdate["physical
PlanDescription"])
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:402)
at
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingExcepti
on.java:361)
at
com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.jav
a:316)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:782)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSer
ializerBase.java:657)
at
com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrapped
Serializer.java:32)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSeri
alizerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(Default
SerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4
624)
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3828)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToJson(JsonProtocol.scala:145)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2(JsonProtocol.scala:66)
at org.apache.spark.util.JsonProtocol$.
$anonfun$writeSparkEventToOutputStream$2$adapted(JsonProtocol.scala:65)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:3206)
at
org.apache.spark.util.JsonProtocol$.writeSparkEventToOutputStream(JsonProtocol.scal
a:65)
at
org.apache.spark.util.JsonProtocolShim$.writeSparkEventToOutputStream(JsonProtocolS
him.scala:16)
at
com.databricks.backend.daemon.driver.DBCEventLoggingListener.sizeLimitedJson(DBCEve
ntLoggingListener.scala:256)
at com.databricks.backend.daemon.driver.DBCEventLoggingListener.
$anonfun$onEvent$1(DBCEventLoggingListener.scala:387)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at
com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(Databr
icksExecutionContext.scala:36)
at com.databricks.threading.NamedExecutor$
$anon$2.$anonfun$run$1(NamedExecutor.scala:367)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.logging.UsageLogging.
$anonfun$withAttributionContext$1(UsageLogging.scala:426)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:196)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:424)
at com.databricks.logging.UsageLogging.withAttributionContext$
(UsageLogging.scala:418)
at
com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:2
94)
at com.databricks.threading.NamedExecutor$
$anon$2.run(NamedExecutor.scala:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.databricks.spark.util.LimitedOutputStream$LimitExceededException:
Exceeded 2097152 bytes (current = 2098441)
at
com.databricks.spark.util.LimitedOutputStream.write(LimitedOutputStream.scala:45)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.ja
va:2203)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGene
rator.java:1515)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGener
ator.java:1462)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGene
rator.java:1345)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.jav
a:517)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.
java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanProperty
Writer.java:733)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSeria
lizerBase.java:774)
... 34 more
25/02/06 06:23:33 INFO AdaptiveSparkPlanExec: Unpersisting Photon join relation
block with ID 683609)
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar at
(spark://10.183.232.44:36145/jars/xml_apis_xml_apis_1_4_01.jar,Some(/local_disk0/
tmp/addedFile391a1c217c474021b46e12a23215cda73099462421691166902/
xml_apis_xml_apis_1_4_01.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:140.
Current active queries:3
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_spark_mongo_spark_connector_2_12_10_1
_1.jar,Some(/local_disk0/tmp/
addedFile2eb15af679934edc8b7394ed2e918f808815439786013148891/
org_mongodb_spark_mongo_spark_connector_2_12_10_1_1.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:137.
Current active queries:2
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:142.
Current active queries:3
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 INFO HashAggregateExec:
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current
version of codegened fast hashmap does not support this aggregate.
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar at
(spark://10.183.232.44:36145/jars/org_apache_xmlbeans_xmlbeans_3_1_0.jar,Some(/
local_disk0/tmp/addedFileddaad57ad7a0487c9bf12f2289b987e38569389366355381576/
org_apache_xmlbeans_xmlbeans_3_1_0.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:142.
Current active queries:2
25/02/06 06:23:33 WARN BlockManager: Asked to remove block join_relation_682561,
which does not exist
25/02/06 06:23:33 WARN BlockManager: Asked to remove block join_relation_685385,
which does not exist
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN BlockManager: Asked to remove block join_relation_684409,
which does not exist
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:138.
Current active queries:1
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:144.
Current active queries:2
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 2, Current Avg
load: 2, New parallelism: 10
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:141.
Current active queries:3
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 10, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:145.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_text_1_8.jar,Some(/
local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:141.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:146.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar at
(spark://10.183.232.44:36145/jars/com_github_virtuald_curvesapi_1_06.jar,Some(/
local_disk0/tmp/addedFile5d83bd5d5cdf484d83304d357af9e9065176332865522817552/
com_github_virtuald_curvesapi_1_06.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:144.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile2f89c9c3b4df458488a3fe7dd78678822232711522257996669/
commons_io_commons_io_2_11_0.jar at
(spark://10.183.232.44:36145/jars/commons_io_commons_io_2_11_0.jar,Some(/
local_disk0/tmp/addedFile2f89c9c3b4df458488a3fe7dd78678822232711522257996669/
commons_io_commons_io_2_11_0.jar)) has been added already. Overwriting of added jar
is not supported in the current version.
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:145.
Current active queries:3
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:143.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 3, Current Avg
load: 2, New parallelism: 7
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN BlockManager: Asked to remove block join_relation_683609,
which does not exist
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:148.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 7, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:147.
Current active queries:5
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 5, Total cores: 20, Current load: 5, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:149.
Current active queries:6
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 6, Current Avg
load: 2, New parallelism: 4
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile0e0d32bfdd08429db5dfcdedd2eef8898265980213250260345/
org_mongodb_bson_4_8_2.jar at
(spark://10.183.232.44:36145/jars/org_mongodb_bson_4_8_2.jar,Some(/local_disk0/
tmp/addedFile0e0d32bfdd08429db5dfcdedd2eef8898265980213250260345/
org_mongodb_bson_4_8_2.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:148.
Current active queries:5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-2ad3d-
cb3a1-348df-3,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar at
(spark://10.183.232.44:36145/jars/org_apache_commons_commons_text_1_8.jar,Some(/
local_disk0/tmp/addedFilef54789c2b4f74a4d92dfda50da605fa0691354238038468595/
org_apache_commons_commons_text_1_8.jar)) has been added already. Overwriting of
added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:143.
Current active queries:4
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar at
(spark://10.183.232.44:36145/jars/com_github_pjfanning_poi_shared_strings_1_0_4.jar
,Some(/local_disk0/tmp/
addedFile7486adfd763e499784763c1bd7f2aac74460962547130545672/
com_github_pjfanning_poi_shared_strings_1_0_4.jar)) has been added already.
Overwriting of added jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:147.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-661af-
0adf4-bffc6-b,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with execution ID:150.
Current active queries:4
25/02/06 06:23:33 INFO AdaptiveParallelism: Updating parallelism using instant
cluster load. Old parallelism: 4, Total cores: 20, Current load: 4, Current Avg
load: 2, New parallelism: 5
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-70d0c-
4a4fd-a2328-a,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar at
(spark://10.183.232.44:36145/jars/commons_codec_commons_codec_1_13.jar,Some(/
local_disk0/tmp/addedFilef38071acf2084a04a797da6c6c35980b7010537528958850384/
commons_codec_commons_codec_1_13.jar)) has been added already. Overwriting of added
jar is not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:146.
Current active queries:3
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 WARN SparkContext: The JAR
file:/local_disk0/tmp/addedFilecaa8bc8ceeed4a32a06581f3937a6b695818999773038281347/
com_h2database_h2_1_4_200.jar at
(spark://10.183.232.44:36145/jars/com_h2database_h2_1_4_200.jar,Some(/local_disk0/
tmp/addedFilecaa8bc8ceeed4a32a06581f3937a6b695818999773038281347/
com_h2database_h2_1_4_200.jar)) has been added already. Overwriting of added jar is
not supported in the current version.
25/02/06 06:23:33 INFO ClusterLoadMonitor: Removed query with execution ID:149.
Current active queries:2
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: current category Some(EXECUTABLE_COMMAND), restoring to
previous category Some(UNDETERMINED).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-7f315-
a9c13-11174-5,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO CurrentQueryContext: Thread Thread[WRAPPER-ReplId-5ea28-
6c7f2-160b6-6,5,main]: Setting current query category as an executable command
(Command is a class org.apache.spark.sql.execution.command.AddJarsCommand).
25/02/06 06:23:33 INFO ClusterLoadMonitor: Added query with exec
You might also like
- Databricks Data Engineer Associate NotesNo ratings yetDatabricks Data Engineer Associate Notes5 pages
- How To Use Badi Ukm - Fill in FSCM: SymptomNo ratings yetHow To Use Badi Ukm - Fill in FSCM: Symptom2 pages
- Logs 2025-01-27T17 53 33.232Z Ri - Foundry.main - Job.No ratings yetLogs 2025-01-27T17 53 33.232Z Ri - Foundry.main - Job.188 pages
- Bcs054 Object Oriented System Design With CNo ratings yetBcs054 Object Oriented System Design With C2 pages
- DSC-3 2932811101 Fundamental of Computer ProgrammingNo ratings yetDSC-3 2932811101 Fundamental of Computer Programming4 pages
- 7 May 2024aws - Replication - Agent - Installer - LogNo ratings yet7 May 2024aws - Replication - Agent - Installer - Log15 pages
- Intro To Apache Spark: Paco Nathan, Download SlidesNo ratings yetIntro To Apache Spark: Paco Nathan, Download Slides86 pages
- FAFP - 16.11.2018 - Hyderabad - Saran Kumar UNo ratings yetFAFP - 16.11.2018 - Hyderabad - Saran Kumar U36 pages
- Laboratory Report of The Basic Arithmetic Operations Using C++0% (1)Laboratory Report of The Basic Arithmetic Operations Using C++8 pages
- (Cbse - Board) Inc SR Cbse Computer - Science - 18 05 2024No ratings yet(Cbse - Board) Inc SR Cbse Computer - Science - 18 05 20245 pages
- Sai Likhith Kanuparthi Resume Mar 19 2020No ratings yetSai Likhith Kanuparthi Resume Mar 19 20201 page
- ENGR 1200 Introduction To Programming Lecture 0No ratings yetENGR 1200 Introduction To Programming Lecture 012 pages
- Azure Data Engineer + Databricks ContentNo ratings yetAzure Data Engineer + Databricks Content7 pages
- Deploying and Managing a Cloud Infrastructure: Real-World Skills for the CompTIA Cloud+ Certification and Beyond: Exam CV0-001From EverandDeploying and Managing a Cloud Infrastructure: Real-World Skills for the CompTIA Cloud+ Certification and Beyond: Exam CV0-001No ratings yet
- Apache Cassandra Administrator Associate - Exam Practice TestsFrom EverandApache Cassandra Administrator Associate - Exam Practice TestsNo ratings yet
