Backup Optim : Primary Key instead of Limit Offset #887
Conversation
GautierBoudeweel
commented
Sep 11, 2024
| Questions | Answers |
|---|---|
| Description? | Please be specific when describing this change. What did you change? Why? |
| Type? | bug fix / improvement / new feature / refactor |
| BC breaks? | yes / no |
| Deprecations? | yes / no |
| Fixed ticket? | Fixes #{issue URL here}, Fixes #{another issue URL here} |
| Sponsor company | Your company or customer's name goes here (if applicable). |
| How to test? | Indicate how to verify that this change works as expected. |
| @@ -219,7 +221,11 @@ public function run(): int | |||
| if (!in_array($table, $ignore_stats_table)) { | |||
| do { | |||
| $backup_loop_limit = $this->container->getState()->getBackupLoopLimit(); | |||
| $data = $this->container->getDb()->executeS('SELECT * FROM `' . $table . '` LIMIT ' . (int) $backup_loop_limit . ',200', false, false); | |||
| if ($primary_key) { | |||
| $data = $this->container->getDb()->executeS('SELECT * FROM `' . $table . '` WHERE `' . $primary_key . '` BETWEEN ' . (int) $backup_loop_limit . ' AND ' . (int) $backup_loop_limit + 200 , false, false); | |||
There was a problem hiding this comment.
Note that in this implementation, if there is a gap in the rows that is bigger than a batch size, nothing would be returned by this request and the sync of the backup would be considered as done.
There was a problem hiding this comment.
We could check for the max id in order to prevent stopping the loop
$maxId = (int)$db->getValue("SELECT MAX($primaryKey) FROM $table");
There was a problem hiding this comment.
That's a great idea, and this should be stored in a cache. We would avoid reloading the same request at each HTTP request.
|
There is another issue with composite primary key, like in ps_cart_rule_combination |