Added user-defined flags to the connector

[poison]

Flags are currently used by the connector to identify the type of the payload, the compression and the serialization. While memcached support at least 16 bits for the flags, only 8 bits are used by the connector. So this leaves us 8 bits to user-land to play with.

The only minor incompatible change is $cas_token in get and getMulti can now be passed by value too instead only by reference. It will only enable CAS if $cas_token is passed by reference.

The following function call won't trigger a warning anymore that parameter 3 should be passed by reference, but instead it won't activate CAS and just return the $udf_flags:

$udf_flags = null;
$memcache->get($key, null, null, $udf_flags);

Please let me know what you think of it. We're using it to enable versioning of keys and we have some other use cases as well.

[mkoppanen]

Let me take a look. There is one test failure in Travis CI, but haven't yet checked why it fails.

[poison]

So basically the thing I wrote about the incompatible change can be discarded since it's fixed in another way.

[mkoppanen]

Ok, I reviewed this patch and I like the functionality. Some comments:

• As this breaks backwards compatibility I think this calls for new major version (3.x)
• The current handling of flags is a bit messy in general:

MEMC_VAL_COMPRESSED and MEMC_VAL_COMPRESSION_ZLIB are two flags where as serialisation types are handled with one flag.

I wonder if this would be a good chance to organise the flags a bit better in general. I've been pondering whether 6 bytes would be enough for user flags as this would allow to reserve 10 bytes for internal flags?

[poison]

I'm not sure if the current patch breaks compatibility (it only adds an extra argument in the getDelayed* return values). The pass-by-value option for &$cas_tokens in the get(Multi)?(byKey)?() function won't throw an error or notice like it did in the past, but you won't be able to use the value from any language construct if you pass by value and not by reference (afaik). (I extended the cas.phpt and cas_multi.phpt to test all the cases).

But I agree that we could save a flag by combining MEMC_VAL_COMPRESSION_ZLIB, MEMC_VAL_COMPRESSION_FASTLZ and MEMC_VAL_COMPRESSED. This would break backwards compatibility.

Starting from memcached 1.2.1, flags is a 32bit integer. 1.2.0 and below it's 16 bit. We could indeed reserve 10 bits internally, this leaves the user space with 6 (or 22) bits. Although I'm not sure how to detect the server version from within the library. For our use-case, 6 bits (int 0-63) is more then enough.

[mkoppanen]

Contemplating a bit further: we could reserver 16 bits for internal use and the remaining 16 bits for user-flags. With user flags we could check the server version before trying to use them (http://docs.libmemcached.org/memcached_version.html?highlight=version) and give error if the server supports only 16bits?

[mkoppanen]

Can you take a look at 'testing-flags' branch under my fork? It seems that this would be a bit cleaner approach to handling type and other flags and should be backwards compatible

[poison]

Ok, I'll check your fork

[poison]

I get the idea, you check the server version of all configured servers whenever udf_flags are used, but I'd also create an ini flag to instruct the connector that we've done our homework and it doesn't need to be checked.

In our production environment I wouldn't like doing this check every time. (And we do use the memcache protocol also for custom services on our website, since we found out that HTTP and REST are too much overhead so internal calls were replaced with memcache. In our case the versioning of the server is different than memcached's versioning.)

I'll have the full code reviewed by this evening.

[mkoppanen]

From libmemcached perspective the returned value is always uint32_t. One option would be to not do the version check and leave that to the user.

[poison]

That's indeed another option, I tried to google it and I see that memcached 1.2.1 was released somewhere in 2007 (didn't find any exact release date)...

[mkoppanen]

i think 6 years is a reasonable expectation for people to update. But then again, if you look at C99 and Visual Studio..

[mkoppanen]

Updated and removed the version check

[mkoppanen]

Found a small backwards compatiblity break. Previously this code worked:

$cas_token = null;
$v = $m->get('cas_test', null, $cas_token);

[mkoppanen]

nm, merged the arginfo changes

[mkoppanen]

Are you happy if I open a PR from testing-flags branch I have? The only major change is that user flags are 16 bits.

[poison]

Yes, a453546 is running in production now and it works great!

[mkoppanen]

Closing this one.

[mkoppanen]

@poison, added you to the php-memcached-dev organisation. Hopefully the invite went to right place.