SQLiteJournal: Persistent connection related problems

[rebendajirijr]

I know this is rather server problem, but how do you handle problems with reusing an existing connection which is made persistent in SQLiteJournal storage by default? I'm facing random errors like
SQLSTATE[HY000]: General error: 1 cannot start a transaction within a transaction or SQLSTATE[HY000]: General error: 10 disk I/O error (mainly on production server which is PHP 5.4.41-0+deb7u1, nginx/1.2.1). The only thing which helps me is switching back to FileJournal.

[milo]

@rebendajirijr Could you send a whole tracy dump? I'm interesed about the stack trace.

[dg]

I found in log errors like:

General error: 8 attempt to write a readonly database in SQLiteJournal.php:71
General error: 5 database is locked in SQLiteJournal.php:64
General error: 1 cannot start a transaction within a transaction in SQLiteJournal.php:61

But I think it was caused by hard deleting file journal.s3db by FTP deployment.

[rebendajirijr]

@dg I'm using FTP deployment, so you think one should not try to delete journal file by using purge[] option?

[mrtnzlml]

There is the very problem if you are using one user for webserver and another user for CLI (Linux). In this case SQLite journal is created and locked by eg. www-data user and if you try to run app from command line you'll get PDOException SQLSTATE[HY000]: General error: 8 attempt to write a readonly database. Also the errors above can be caused by different system users (again created by webserver and impossible to use by system user because it's blocked).

Is there some kind of workaround? I know it's possible to use FileJournal, but I don't think this is solution. Also I think this server setup is not that wrong. cc @milo

[milo]

@mrtnzlml SQLite3 creates database with 0644 mode as default (umask 0) and cannot be changed (or I didn't find the way). To be consistent with FileJournal (fopen()'s default is 0666), workaround will be needed.

Anyway, if you can, use filesystem's ACL like:

setfacl -m 'd:u:www-data:rwx' temp
setfacl -m 'd:u:mrtnzlml:rwx' temp

and you are out of trouble.

[Daaarkling]

Hi, same trouble right here but @milo trick did the job.

[janedbal]

I'm facing this issue when deploying by FTP deployment and deleting the file. After deploy, I get error 500 with one of errors mentioned by @dg. What is the correct workflow when using such tool? I should not delete the file (I would expect that all files in temp/cache are deletable)?

Deployment runs as the same user as nginx uses for handling requests.

[milo]

Maybe it's an issue for persistent connections only. But it is just an idea. Could you try to remove pesistency here?

[janedbal]

Yes, the persistence is the problem. Without that, everything works as expected. It is probably caused by reusing handles as mentioned in note here.

[janedbal]

@milo Do you think we should remove the persistence? Should I send PR?

[milo]

I think that it can be non-persistent. PR is welcomed.

Measured with:

Apache 2.4 on Win7, PHP 5.6.13 as an Apache module

# Connect only
$journal = new SQLiteJournal(__DIR__ . '/persistency.db');

$ ab -c 50 -n 10000 http://localhost.56m/dev/nette/caching.tests/sqlite-persistency.php

Connection Times (ms)
              min  mean[+/-sd] median   max
# Persistent (connect only)
Connect:        0    0   0.3      0       1
Processing:     5   31   9.5     31     102
Waiting:        5   31   9.5     31     102
Total:          5   32   9.5     31     102

# Not persistent (connect only)
Connect:        0    0   0.3      0       1
Processing:     4   36  12.2     35     131
Waiting:        4   36  12.2     35     131
Total:          4   36  12.2     36     132

# + write / clean
foreach (range(0, 1000) as $i) { $journal->write("key-$i", []); }
$journal->clean([Cache::ALL]);

$ ab -c 50 -n 1000 http://localhost.56m/dev/nette/caching.tests/sqlite-persistency.php

# Persistent (write/clean)
Connect:        0    0   0.3      0       1
Processing:   554  733  24.6    734     827
Waiting:      554  733  24.6    734     827
Total:        554  733  24.6    734     827

# Not persistent (write/clean)
Connect:        0    0   0.3      0       1
Processing:   555  735  22.4    736     812
Waiting:      555  735  22.4    736     812
Total:        555  735  22.4    736     812

[JanTvrdik]

👍 For not using persistent connection by default because currently I need to restart web server in order to delete cache.

[dg]

Closed by #32