jsoncpp-devel Mailing List for json-cpp

On Tue, May 24, 2011 at 11:57 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> Yes I think we should go ahead. The current behavior of having the isXYZ
> method returning true and asXYZ failing with an exception is certainly NOT
> what a user would expect.

Okay, I'll get started on this today.

Ah right, I see. I actually fixed this by adding braces in the SVN
commit I made, but if you want me to add a function I can.

On Wed, May 25, 2011 at 1:43 AM, Baptiste Lepilleur
<bap...@gm...> wrote:
>
>
> 2011/5/24 Aaron Jacobs <ja...@go...>
>>
>> On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
>> <bap...@gm...> wrote:
>> [...]
>> > - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
>> > multi-statements). IMHO it should just call a function that dump message
>> > and
>> > then call abort().
>>
>> Sorry, I'm not sure I understand the first sentence here. Could you
>> clarify?
>
> Sorry, I should have made this clearer:
> #define JSON_FAIL_MESSAGE( message ) std::cerr << message; exit(123);
> with:
> if (!ok) JSON_FAIL_MESSAGE(reader.getFormattedErrorMessages());
> will becomes:
> if (!ok)
> std::cerr << message;
> exit(123);
> A simple way to avoid this is:
> #define JSON_FAIL_MESSAGE( message ) onJsonFailure( message );
> Baptiste.
>

2011/5/24 Aaron Jacobs <ja...@go...>

> On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> [...]
> > - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
> > multi-statements). IMHO it should just call a function that dump message
> and
> > then call abort().
>
> Sorry, I'm not sure I understand the first sentence here. Could you
> clarify?
>

Sorry, I should have made this clearer:
#define JSON_FAIL_MESSAGE( message ) std::cerr << message; exit(123);
with:
if (!ok) JSON_FAIL_MESSAGE(reader.getFormattedErrorMessages());

will becomes:
if (!ok)
std::cerr << message;
exit(123);

A simple way to avoid this is:
#define JSON_FAIL_MESSAGE( message ) onJsonFailure( message );

Baptiste.

2011/5/24 Aaron Jacobs <ja...@go...>

> On Tue, May 24, 2011 at 8:34 PM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> > This is what I had in mind when I said that "isUInt() should succeed if
> the
> > value is a positive signed int Value". I think it moves the API in the
> right
> > direction: there should always be a way to know if a function will throw
> > before calling it.
>
> I like it. Should it be done?
>

Yes I think we should go ahead. The current behavior of having the isXYZ
method returning true and asXYZ failing with an exception is certainly NOT
what a user would expect.

Looking back at the existing API, it is likely that user to code like
isInt() || isUInt() to get a positive integer as it is difficult to predict
if the value 1 is a int or uint...

Hopefully it will not break existing code, but this change can be clearly
documented in the NEWS file. There are some unit tests for the isXYZ
functions, so we will see exactly what is impacted.


> > That being said, when I look back at your code sample:
> >    unsigned int value =
> >        value.isUInt() ? value.asUInt() :
> >            (value.isDouble() ? value.asDouble() : kError);
> >
> > I can not help but feel that something is wrong. If value does not fit on
> a
> > 32 bits integer, then you will get mostly a random value after conversion
> to
> > double followed by a cast to unsigned int. Am I missing something?
>
> Ah, good point. The real original code stored the result in int64, and
> happened to know that only values less than 2^63 would be consumed as
> input. The 'unsigned int' bit is an error in my transcription.
>

OK, I though it would be something along those lines.

Baptiste.


> Aaron
>

On Tue, May 24, 2011 at 8:34 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> This is what I had in mind when I said that "isUInt() should succeed if the
> value is a positive signed int Value". I think it moves the API in the right
> direction: there should always be a way to know if a function will throw
> before calling it.

I like it. Should it be done?


> It is not too late. It should not be difficult to add a data member to
> Features instance passed to the Reader to control its behavior that would
> indicate if integer greater than 32 bits should be stored as double or int
> in the Json::Value. Feel free to add this behavior, it should not be too
> difficult.

If we choose the solution above, then this isn't really necessary,
except for strict backwards compatibility. I'll take a look and see if
it's actually necessary at Google as a case study.


> That being said, when I look back at your code sample:
>    unsigned int value =
>        value.isUInt() ? value.asUInt() :
>            (value.isDouble() ? value.asDouble() : kError);
>
> I can not help but feel that something is wrong. If value does not fit on a
> 32 bits integer, then you will get mostly a random value after conversion to
> double followed by a cast to unsigned int. Am I missing something?

Ah, good point. The real original code stored the result in int64, and
happened to know that only values less than 2^63 would be consumed as
input. The 'unsigned int' bit is an error in my transcription.

Aaron

2011/5/23 Aaron Jacobs <ja...@go...>

> On Mon, May 23, 2011 at 9:50 PM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> > If I understood your problem correctly, this should solve your backward
> > compatibility issue. Is that correct?
>
> Almost but not quite, I think. Now the numbers in [2^32, 2^64) will be
> parsed as uint64s, and the code above will always return kError
> because both isUInt and isDouble return false, even though it used to
> work for those values. I think maybe the only choice here is to have
> each of the isFoo methods return true iff asFoo will work, whether
> that's the best way to represent the number or not. For example:
>
> 0.7 -> isDouble
> 1 -> isUInt, isInt, isUInt64, isInt64, isDouble
> 2^32 -> isUInt, isUInt64, isDouble
> 2^40 -> isUInt64, isDouble
> 2^64 - 1 -> isDouble
> 2^64 -> isDouble
>
> I can imagine that this change would cause problems for other existing
> code, but I'm not sure. What do you think?
>

This is what I had in mind when I said that "isUInt() should succeed if the
value is a positive signed int Value". I think it moves the API in the right
direction: there should always be a way to know if a function will throw
before calling it.

In term of backward compatibility I would not expect too many issue. The
most impacting change will likely be that 1 will return true for all
numeric isUInt, isInt, isUInt64, isInt64, isDouble while before it only
returned true for either isUInt or isInt depending on how it was
constructed. When you look at the current behavior, it really sound buggy...

I'm curious has existing code that would break with such change to the
semantic of isInt() and isUInt()...


>
> Perhaps the best way would have been to have users explicitly opt in
> for 64-bit support. That is, add an option to the constructor of
> Json::Reader that defaults to false, and if you set it to true then
> you're asserting that your code doesn't suffer from the problem
> mentioned above. I guess it's too late for that, though.
>

It is not too late. It should not be difficult to add a data member to
Features instance passed to the Reader to control its behavior that would
indicate if integer greater than 32 bits should be stored as double or int
in the Json::Value. Feel free to add this behavior, it should not be too
difficult.

That being said, when I look back at your code sample:
   unsigned int value =
       value.isUInt() ? value.asUInt() :
           (value.isDouble() ? value.asDouble() : kError);

I can not help but feel that something is wrong. If value does not fit on a
32 bits integer, then you will get mostly a random value after conversion to
double followed by a cast to unsigned int. Am I missing something?

Baptiste.

2011/5/23 Stephan Beal <sg...@go...>

> On Mon, May 23, 2011 at 1:50 PM, Baptiste Lepilleur <
> bap...@gm...> wrote:
>
>> I propose making the following changes:
>> 1) isUInt() returns true only asUInt() can succeed. Similar changes for
>> isInt().
>>     I think this contract should be generalized. e.g. asUInt() should
>> succeed if the value is a positive signed int Value.
>> 2) Adds new member functions:
>>    isUInt64() that returns true only if asUInt64() can succeed.
>>    and isInt64()...
>>
>> If I understood your problem correctly, this should solve your backward
>> compatibility issue. Is that correct?
>>
>
> My proposal would be to remove the int/uint distinction entirely, since
> JavaScript and JSON do not distinguish between the two. In my own JSON code
> i simply use int64_t for all integers and double for doubles (though JS
> there's just a single Number type, if i'm not mistaken). In my own use of
> json libs and JS engines (SpiderMonkey, QtScript, and Google v8), i've
> always found the distinction between signed/unsigned to be philosophically
> unsettling because JS doesn't natively have unsigned numbers.
>

In fact I think if we were just interested in supporting Javascript
compatibility we could probably get away with supporting only double.

There was an intermediate version of jsoncpp where the only integers type
were int64 / uint64 but this kind of change broke backward compatibility
painfully, for example causing printf( "%d", value.asInt()) to behave in
"interesting" way.

Also one of the aim of JsonCpp is for use for configuration file. In such
case you frequently need to deal with int/ unsigned int types. If the only
getter available were 64 bits integer we would be precision loss warning all
over the place.

Though if you use asUInt64() you can easily pretend that the other integer
type do not exist (minor the identified lack of isUInt64() with the right
semantic).


>  JSON, in fact, does not specify the precision of numeric values (
> http://www.ietf.org/rfc/rfc4627.txt?number=4627), probably because it
> would be impossible to guaranty that any given implementation (in umpteen[1]
> different programming languages) can honor that, e.g. a certain embedded
> environment might not have integers >32 bits. Section 2.4 of the JSON RFC
> notably does not specify any limits on numbers, so 2^75 (when written in
> expanded integer form) is syntactically legal, according to the grammar, but
> is almost certainly semantically illegal in any modern computer.
>
These pages:
>
> http://www.hunlock.com/blogs/The_Complete_Javascript_Number_Reference
>  http://www.jibbering.com/faq/#FAQ4_7
>
> say that in JS integers are reliable up to "15-16" digits (53 bits). Thus
> int64_t can legally hold any JS-legal value.
>
> Obviously, JSON is _not_ only for JavaScript, but i'm using JS as the
> baseline here because (A) JSON derives from JS and (B) it is primarily
> consumed by JS applications (though it is primarily generated by other
> technologies).
>
>
Actually, some parser leverage language features that bypass hardware
limitation. Python will happily parse a 2^75 integer. Support for use of the
arbitrary floating point precision type Decimal was even added at some point
in python. So there is clearly use cases around there where people just use
JSON for serialization and don't limit themselves to what can be done
portably.

Baptiste.


> [1] = American slang for "very many"
>
>

On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
<bap...@gm...> wrote:
> To make sure I understand the issue: sscanf on Mac OS X will write in the
> format string provided as parameter even though it is declared const char *?

That's right. Don't ask me. :-/

> - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
> multi-statements). IMHO it should just call a function that dump message and
> then call abort().

Sorry, I'm not sure I understand the first sentence here. Could you clarify?


> - partial duplication of assertions macros already present in json_value.cpp
>
> I think we should move the assertion/failure macros from json_value.cpp  to
> a common private header instead of duplicating them.
> In the future, we may want to provide hook to allow user to customize what
> happens on failure (e.g. like google breakpad integration or whatever stack
> capture tool you happen to use...)

Okay, sounds good.