jsoncpp-devel Mailing List for json-cpp (Page 2)

amalgamate.py currently doesn't work on Linux because some paths are
done using backslashes instead of forward slashes. I think the Right
Way (TM) to do this is to use os.path, but here's a lazy patch that
just swaps the slashes.

-Joe G.

Json-C (http://oss.metaparadigm.com/json-c/) also installs file
/usr/include/json/json.c. Json-C package is present at least in
some Linux distributions (it is in Fedora)

It would be nice to avoid conflict with JsonCpp header file
include/json/json.h. One way would be to move all headers to
include/jsoncpp/ from include/json/. Unfortunatelly this
change would break already existing software which uses
JsonCpp.

Perhaps it would be better to put headers in new place and
add forwarding headers in old place. For example:

include/json/json.h could contain

#ifndef __JSONCPP_JSON_JSON_H__
#define __JSONCPP_JSON_JSON_H__
#warning Header file json/json.h is deprecated. Use jsoncpp/json.h instead
#include <jsoncpp/json.h>
#endif

and similarly for other headers. After some time it would then be possible
to remove headers at old place.

Andris

On Fri, May 27, 2011 at 10:37 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> I'm done fixing most portability issues. It now compiles and pass all tests
> on:
> IBM AIX on Power6 with XLC 7
> Solaris X86 / Sunstudio 12
> Solaris SPARC / Sunstudio 12
> Windows XP SP3 / MSVS 6.6, 2003, 2005, 2008
> Linux redhat 3 / gcc 3.2.3
> Linux redhat5. 3 / gcc 4.1.2
> There is still some failing unit tests with MSVS 2010 that I need to
> investigate...

Awesome, thanks so much for cleaning up after this. Out of curiosity:
do you actually have an AIX machine to test with? Wow.

2011/5/27 Aaron Jacobs <ja...@go...>

> Wow, that's crazy. I think your solution to the first problem is good.
> For the second, maybe the best would be to *parse* the floating point
> string to be checked, checking that it parses successfully and differs
> from the original double by no more than 0.1% or whatever?
>

I've went ahead and did the "normalize exponent string" approach since
comparing double with ratio is always tricky (e.g; 0.1% of which number...).

I'm done fixing most portability issues. It now compiles and pass all tests
on:
IBM AIX on Power6 with XLC 7
Solaris X86 / Sunstudio 12
Solaris SPARC / Sunstudio 12
Windows XP SP3 / MSVS 6.6, 2003, 2005, 2008
Linux redhat 3 / gcc 3.2.3
Linux redhat5. 3 / gcc 4.1.2

There is still some failing unit tests with MSVS 2010 that I need to
investigate...


>
> On Thu, May 26, 2011 at 5:45 PM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> >
> >
> > 2011/5/25 Aaron Jacobs <ja...@go...>
> >>
> >> Hmm, but does it make sense to say a default constructed value is all
> >> of null, an array, and an object? Wouldn't it make more sense to have
> >> isArray (and only isArray) start returning true once the value is
> >> "turned into" an array by appending something to it?
> >>
> >> I'd appreciate if you could investigate the VC++ issues; I don't have
> >> easy access to a Windows machine.
> >
> > The unit test failures are real strange:
> >    double x = kint64max;
> >    double y = val.asDouble();
> >    int count = 0;
> >    if ( y == x ) // OK (case 1)
> >    {
> >        ++count;
> >    }
> >    if ( double(kint64max) == y ) // BAD (case 2)
> >    {
> >        ++count;
> >    }
>
[...]
>

I though I would share the portability issues I run into and fixed the last
few days:

- Do not use STL API that requires special handling of 64 bits integers.
Those can lead to interesting errors (MSVC 6 is a typical example):
  My understanding is that the initial STL standard did not acknoledge the
existence of 64 bits integers and STL implementor did not extend the API on
their own...
  - std::ostream do not support 64 bits integer, leading to ambiguous
overload compilation error
  - std::numeric_limits<> compiles but returns 0 at run_time
- Do not rely on formatting of floating point numbers as string:
  MSVC typically always output 3 digits in the exponent, Unix platforms have
a varying number of digits
  (This was solved by introducing the normalizeFloatingPointStr() function
to normalize the formatting of the floating-point exponent)
- uint64 to double conversion is trouble:
  Unfortunately no generic recipes. Requires multi-platform testing when
introducing new use cases...
  - not supported by MSVC6 (worked-around by using int64 to double
conversion)
  - can run into interesting precision issues when comparing double stored
in local variable and FPU registers (that may be of greater precision)
  - some strange issue when converting uint64 max to double on the very old
XLC 7 compiler (produced -1...)
- Do not use LL and ULL for 64 bits literal
  Unfortunately not supported by all compilers (MSVC 6 for instance).
  - work-around this by converting an integer to int64. Example: UInt64(1)
<< 63

Baptiste.

Wow, that's crazy. I think your solution to the first problem is good.
For the second, maybe the best would be to *parse* the floating point
string to be checked, checking that it parses successfully and differs
from the original double by no more than 0.1% or whatever?

On Thu, May 26, 2011 at 5:45 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
>
>
> 2011/5/25 Aaron Jacobs <ja...@go...>
>>
>> Hmm, but does it make sense to say a default constructed value is all
>> of null, an array, and an object? Wouldn't it make more sense to have
>> isArray (and only isArray) start returning true once the value is
>> "turned into" an array by appending something to it?
>>
>> I'd appreciate if you could investigate the VC++ issues; I don't have
>> easy access to a Windows machine.
>
> The unit test failures are real strange:
>    double x = kint64max;
>    double y = val.asDouble();
>    int count = 0;
>    if ( y == x ) // OK (case 1)
>    {
>        ++count;
>    }
>    if ( double(kint64max) == y ) // BAD (case 2)
>    {
>        ++count;
>    }
> Looking at the assembly, the only difference I see is that in the first case
> the x value (kint64max) is loaded from the local variable, while in the
> second case it never leave the FPU register if I understand correctly. My
> guess is that we are stumbling into the fact that FPU register have an
> internal precision of 80 bits. I'm not sure how to confirm this though...
> Case 1:
>    if ( y == x )
> 00411B97  fld         qword ptr [ebp-290h]
> 00411B9D  fcomp       qword ptr [ebp-2A0h]
> 00411BA3  fnstsw      ax
> Case 2:
>    if ( double(kint64max) == y )
> 00411BB9  call        std::numeric_limits<__int64>::max (412D60h)
> 00411BBE  mov         dword ptr [ebp-0A4Ch],eax
> 00411BC4  mov         dword ptr [ebp-0A48h],edx
> 00411BCA  fild        qword ptr [ebp-0A4Ch]
> 00411BD0  fcomp       qword ptr [ebp-2A0h]
> 00411BD6  fnstsw      ax
> I worked-around this by making code match case 1...
>
> There is another kinds of test failures that occurs related to how floating
> pointer number are represented as string. With MSVS AFAIK we always get
> three digits in the exponent. Here are some examples of failures:
> 1>* Detail of ValueTest/integers test failure:
> 1>..\..\src\test_lib_json\main.cpp(582): "1.04858e+06" == val.asString()
> 1>  Expected: '1.04858e+06'
> 1>  Actual  : '1.04858e+006'
>
> * Test E:\prg\vc\Lib\jsoncpp-trunk\test\data\test_real_09.json
>   Difference in input at line 1:
>   Expected: '.=1.9e+19'
>   Actual:   '.=1.9e+019'
> The easiest is probably to normalize floating point string before
> comparison. What do you think?
> Baptiste.

On Wed, May 25, 2011 at 10:52 PM, Aaron Jacobs <ja...@go...> wrote:

> Hmm, but does it make sense to say a default constructed value is all
> of null, an array, and an object? Wouldn't it make more sense to have
>

In JS, default-constructed values are 'undefined', not null:

[stephan@cheyenne:~/tmp]$ js
Rhino 1.7 release 2 2010 11 17
js> var x; print(typeof x);
undefined

(And (typeof null) == object, for some brain-dead reason or other.)

IMO that is the proper default-construction semantics. The further a JSON
impl deviates from basic JS behaviours, the less portable it is likely to
become.


isArray (and only isArray) start returning true once the value is
> "turned into" an array by appending something to it?
>

Using JS as a baseline:

js> x[1] = 3;
js: "<stdin>", line 3: uncaught JavaScript runtime exception: TypeError:
Cannot set property "1.0" of undefined to "3"
at <stdin>:3

that's the behaviour most would expect, i think (as opposed to automatic
promotion to an Array or Object).

-- 
----- stephan beal
http://wanderinghorse.net/home/stephan/

2011/5/25 Stephan Beal <sg...@go...>

> On Wed, May 25, 2011 at 4:56 PM, Baptiste Lepilleur <
> bap...@gm...> wrote:
>
>>
>>> This is that feature that allows the following usage:
>>
>> Json::Value array;
>> array.append( 1234 );
>>
>
> That means a null value is effectively mutable, and only arrays/objects are
> mutable in JS (JSON doesn't describe mutability, and it has no operators
> which would allow mutation).
>

The Json::Value is mutable and provides and operator = (Json::Value is not
supposed to be dynamically allocated). There is a Json::Value::null
constant, but its usage is not enforced at the current time.


>
> Maybe we should introduce a distinction between "default" initialization
>> and "null", but I'm not sure how to go about that...
>>
>
> Absolutely. In the 2 JSON libs i've implemented (one in C++, one in C), the
> following values all have a corresponding static/constant Value instance
> (mainly as a memory allocation optimization):
>

The cost of creating a Json::Value instance is fairly small since for all
types with the exception of string/array/object it is just an enum and
primitive type assignment. Json::Value is just a wrapper on top of a
discriminated union.


> [...]
>
> A default-constructed Value has the undefined value:
>
> Value v; // v.isUndef() will return true
> v = Object(); // now v.isObject() will return true
>

This is something that we will need to investigate in the future. I think
that making the distinction between default construction and null value make
sense. But at the current I want to focus on making sure that 64 bits
integer support is in good shape before making other API changes.

Thanks for you feedback, it's interesting to see what other API are around
there.

Baptiste.

2011/5/25 Aaron Jacobs <ja...@go...>

> Hmm, but does it make sense to say a default constructed value is all
> of null, an array, and an object? Wouldn't it make more sense to have
> isArray (and only isArray) start returning true once the value is
> "turned into" an array by appending something to it?
>
> I'd appreciate if you could investigate the VC++ issues; I don't have
> easy access to a Windows machine.
>

The unit test failures are real strange:

   double x = kint64max;
   double y = val.asDouble();
   int count = 0;
   if ( y == x ) // OK (case 1)
   {
       ++count;
   }
   if ( double(kint64max) == y ) // BAD (case 2)
   {
       ++count;
   }

Looking at the assembly, the only difference I see is that in the first case
the x value (kint64max) is loaded from the local variable, while in the
second case it never leave the FPU register if I understand correctly. My
guess is that we are stumbling into the fact that FPU register have an
internal precision of 80 bits. I'm not sure how to confirm this though...

Case 1:
   if ( y == x )
00411B97  fld         qword ptr [ebp-290h]
00411B9D  fcomp       qword ptr [ebp-2A0h]
00411BA3  fnstsw      ax

Case 2:
   if ( double(kint64max) == y )
00411BB9  call        std::numeric_limits<__int64>::max (412D60h)
00411BBE  mov         dword ptr [ebp-0A4Ch],eax
00411BC4  mov         dword ptr [ebp-0A48h],edx
00411BCA  fild        qword ptr [ebp-0A4Ch]
00411BD0  fcomp       qword ptr [ebp-2A0h]
00411BD6  fnstsw      ax

I worked-around this by making code match case 1...


There is another kinds of test failures that occurs related to how floating
pointer number are represented as string. With MSVS AFAIK we always get
three digits in the exponent. Here are some examples of failures:

1>* Detail of ValueTest/integers test failure:
1>..\..\src\test_lib_json\main.cpp(582): "1.04858e+06" == val.asString()
1>  Expected: '1.04858e+06'
1>  Actual  : '1.04858e+006'


* Test E:\prg\vc\Lib\jsoncpp-trunk\test\data\test_real_09.json
  Difference in input at line 1:
  Expected: '.=1.9e+19'
  Actual:   '.=1.9e+019'

The easiest is probably to normalize floating point string before
comparison. What do you think?

Baptiste.

Okay, in revision 224 I've further reworked the type system to act as follows:

 *  isFoo methods determine exact representability.
 *  asFoo methods cause casting when safe.
 *  isConvertibleTo indicates whether casting is safe.

(See NEWS.txt for more details.) I think this makes isConvertibleTo fit in
with the new system properly. I had for example found code at Google that
looks like this:

    int ConvertValueToInt(const Json::Value& v) {
      if (v.isConvertibleTo(Json::intValue)) {
        return v.asInt();
      }
      else { ...}
    }

That caused problems with my last attempt because asInt() would chuck an
exception if the value was 3.5, for example. Now I've restored the behavior of
casting when requested.

The remaining rough edge is 64-bit integers -- it would break existing code to
introduce a new ValueType enum value for them, and given the code above we
can't have `Value(1LL<<40).isConvertibleTo(intValue)` return true. So as of
now, `isConvertibleTo(intValue)` may return false even if the value's type is
intValue. (See NEWS.txt for a better explanation.)

Hmm, but does it make sense to say a default constructed value is all
of null, an array, and an object? Wouldn't it make more sense to have
isArray (and only isArray) start returning true once the value is
"turned into" an array by appending something to it?

I'd appreciate if you could investigate the VC++ issues; I don't have
easy access to a Windows machine.

On Wed, May 25, 2011 at 4:56 PM, Baptiste Lepilleur <
bap...@gm...> wrote:

>
>> This is that feature that allows the following usage:
>
> Json::Value array;
> array.append( 1234 );
>

That means a null value is effectively mutable, and only arrays/objects are
mutable in JS (JSON doesn't describe mutability, and it has no operators
which would allow mutation).


> Maybe we should introduce a distinction between "default" initialization
> and "null", but I'm not sure how to go about that...
>

Absolutely. In the 2 JSON libs i've implemented (one in C++, one in C), the
following values all have a corresponding static/constant Value instance
(mainly as a memory allocation optimization):

true
false
null
"" (empty string)
undefined (not used by JSON, but as a placeholder for default-initialized
values)
double 0.0
integer 0

(the last two are internal optimizations, and not generally all that useful)

A default-constructed Value has the undefined value:

Value v; // v.isUndef() will return true
v = Object(); // now v.isObject() will return true

If someone wants a Null value, they get a reference to the shared instance:

Value v = Value::Null();

This requires no allocation (the object instances are static/const), and
because values are all immutable (except array/object, of course), this is
safe in terms of "nobody can change the value of Null by assigning over a
reference to it."

i don't know if the jsoncpp code is structured such that such constant
values would be immediately implementable, though.

-- 
----- stephan beal
http://wanderinghorse.net/home/stephan/

2011/5/25 Aaron Jacobs <ja...@go...>

> Alright, I've implemented my understanding of the desired behavior in SVN
> revision 216, documenting the change in NEWS.txt.
>
> A few questions:
>
>  *  Could you take a look at NEWS.txt and tell me whether you agree with
> the
>    behavior? I made changes to the other isFoo methods to make them more
>    consistent and less surprising (for example, I don't know why null
>    returned true for isArray).
>

Concerning numeric it looks good. I'll make a more in depth review this
evening.

The behavior of isArray can be explained by the fact that a default
constructed Value can be though of as either an empty array or an empty
object: a nullValue is automatically "upgraded" to arrayValue or objectValue
on first array access or member reference. There is no distinction in the
(current) implementation between a default constructed value and a
nullValue.

This is that feature that allows the following usage:

Json::Value array;
array.append( 1234 );

Json::Value object;
object["age"] = 1234;

Maybe we should introduce a distinction between "default" initialization and
"null", but I'm not sure how to go about that...


>  *  It feels like now that these completely describe the safety of the
> asFoo
>    methods, the isConvertibleTo method is redundant. It's also not
> completely
>    descriptive, since it doesn't distinguish 64-bit integers. Should I
> delete
>    it? We don't ever use it at Google, but I suppose this may be a problem
>    for other codebases.
>

It is best not to break the code unless we have to. We can mark it as
deprecated and have the implementation forward to is* methods (64 variants
for int).

Notes that I have test failures on MSVS2010:
1>  * Detail of ValueTest/integers test failure:
1>  c:\perso\prg\vc\lib\jsoncpp-trunk\src\test_lib_json\main.cpp(626):
double(kint64max) == val.asDouble()
1>  c:\perso\prg\vc\lib\jsoncpp-trunk\src\test_lib_json\main.cpp(627):
float(kint64max) == val.asFloat()
1>  16/17 tests passed (1 failure(s))

Do you know what could be the cause? Otherwise I'll investigate...

There is also the problem of stdint.h which is not available before
MSVS2010:
src\test_lib_json\main.cpp(6) : fatal error C1083: Cannot open include file:
'stdint.h': No such file or directory
=> I'll check exact usage and see how to solve this issue for portability.

Baptiste.


>
> Aaron
>

Alright, I've implemented my understanding of the desired behavior in SVN
revision 216, documenting the change in NEWS.txt.

A few questions:

 *  Could you take a look at NEWS.txt and tell me whether you agree with the
    behavior? I made changes to the other isFoo methods to make them more
    consistent and less surprising (for example, I don't know why null
    returned true for isArray).

 *  It feels like now that these completely describe the safety of the asFoo
    methods, the isConvertibleTo method is redundant. It's also not completely
    descriptive, since it doesn't distinguish 64-bit integers. Should I delete
    it? We don't ever use it at Google, but I suppose this may be a problem
    for other codebases.

Aaron

On Tue, May 24, 2011 at 11:57 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> Yes I think we should go ahead. The current behavior of having the isXYZ
> method returning true and asXYZ failing with an exception is certainly NOT
> what a user would expect.

Okay, I'll get started on this today.

Ah right, I see. I actually fixed this by adding braces in the SVN
commit I made, but if you want me to add a function I can.

On Wed, May 25, 2011 at 1:43 AM, Baptiste Lepilleur
<bap...@gm...> wrote:
>
>
> 2011/5/24 Aaron Jacobs <ja...@go...>
>>
>> On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
>> <bap...@gm...> wrote:
>> [...]
>> > - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
>> > multi-statements). IMHO it should just call a function that dump message
>> > and
>> > then call abort().
>>
>> Sorry, I'm not sure I understand the first sentence here. Could you
>> clarify?
>
> Sorry, I should have made this clearer:
> #define JSON_FAIL_MESSAGE( message ) std::cerr << message; exit(123);
> with:
> if (!ok) JSON_FAIL_MESSAGE(reader.getFormattedErrorMessages());
> will becomes:
> if (!ok)
> std::cerr << message;
> exit(123);
> A simple way to avoid this is:
> #define JSON_FAIL_MESSAGE( message ) onJsonFailure( message );
> Baptiste.
>

2011/5/24 Aaron Jacobs <ja...@go...>

> On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> [...]
> > - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
> > multi-statements). IMHO it should just call a function that dump message
> and
> > then call abort().
>
> Sorry, I'm not sure I understand the first sentence here. Could you
> clarify?
>

Sorry, I should have made this clearer:
#define JSON_FAIL_MESSAGE( message ) std::cerr << message; exit(123);
with:
if (!ok) JSON_FAIL_MESSAGE(reader.getFormattedErrorMessages());

will becomes:
if (!ok)
std::cerr << message;
exit(123);

A simple way to avoid this is:
#define JSON_FAIL_MESSAGE( message ) onJsonFailure( message );

Baptiste.

2011/5/24 Aaron Jacobs <ja...@go...>

> On Tue, May 24, 2011 at 8:34 PM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> > This is what I had in mind when I said that "isUInt() should succeed if
> the
> > value is a positive signed int Value". I think it moves the API in the
> right
> > direction: there should always be a way to know if a function will throw
> > before calling it.
>
> I like it. Should it be done?
>

Yes I think we should go ahead. The current behavior of having the isXYZ
method returning true and asXYZ failing with an exception is certainly NOT
what a user would expect.

Looking back at the existing API, it is likely that user to code like
isInt() || isUInt() to get a positive integer as it is difficult to predict
if the value 1 is a int or uint...

Hopefully it will not break existing code, but this change can be clearly
documented in the NEWS file. There are some unit tests for the isXYZ
functions, so we will see exactly what is impacted.


> > That being said, when I look back at your code sample:
> >    unsigned int value =
> >        value.isUInt() ? value.asUInt() :
> >            (value.isDouble() ? value.asDouble() : kError);
> >
> > I can not help but feel that something is wrong. If value does not fit on
> a
> > 32 bits integer, then you will get mostly a random value after conversion
> to
> > double followed by a cast to unsigned int. Am I missing something?
>
> Ah, good point. The real original code stored the result in int64, and
> happened to know that only values less than 2^63 would be consumed as
> input. The 'unsigned int' bit is an error in my transcription.
>

OK, I though it would be something along those lines.

Baptiste.


> Aaron
>

On Tue, May 24, 2011 at 8:34 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> This is what I had in mind when I said that "isUInt() should succeed if the
> value is a positive signed int Value". I think it moves the API in the right
> direction: there should always be a way to know if a function will throw
> before calling it.

I like it. Should it be done?


> It is not too late. It should not be difficult to add a data member to
> Features instance passed to the Reader to control its behavior that would
> indicate if integer greater than 32 bits should be stored as double or int
> in the Json::Value. Feel free to add this behavior, it should not be too
> difficult.

If we choose the solution above, then this isn't really necessary,
except for strict backwards compatibility. I'll take a look and see if
it's actually necessary at Google as a case study.


> That being said, when I look back at your code sample:
>    unsigned int value =
>        value.isUInt() ? value.asUInt() :
>            (value.isDouble() ? value.asDouble() : kError);
>
> I can not help but feel that something is wrong. If value does not fit on a
> 32 bits integer, then you will get mostly a random value after conversion to
> double followed by a cast to unsigned int. Am I missing something?

Ah, good point. The real original code stored the result in int64, and
happened to know that only values less than 2^63 would be consumed as
input. The 'unsigned int' bit is an error in my transcription.

Aaron

2011/5/23 Aaron Jacobs <ja...@go...>

> On Mon, May 23, 2011 at 9:50 PM, Baptiste Lepilleur
> <bap...@gm...> wrote:
> > If I understood your problem correctly, this should solve your backward
> > compatibility issue. Is that correct?
>
> Almost but not quite, I think. Now the numbers in [2^32, 2^64) will be
> parsed as uint64s, and the code above will always return kError
> because both isUInt and isDouble return false, even though it used to
> work for those values. I think maybe the only choice here is to have
> each of the isFoo methods return true iff asFoo will work, whether
> that's the best way to represent the number or not. For example:
>
> 0.7 -> isDouble
> 1 -> isUInt, isInt, isUInt64, isInt64, isDouble
> 2^32 -> isUInt, isUInt64, isDouble
> 2^40 -> isUInt64, isDouble
> 2^64 - 1 -> isDouble
> 2^64 -> isDouble
>
> I can imagine that this change would cause problems for other existing
> code, but I'm not sure. What do you think?
>

This is what I had in mind when I said that "isUInt() should succeed if the
value is a positive signed int Value". I think it moves the API in the right
direction: there should always be a way to know if a function will throw
before calling it.

In term of backward compatibility I would not expect too many issue. The
most impacting change will likely be that 1 will return true for all
numeric isUInt, isInt, isUInt64, isInt64, isDouble while before it only
returned true for either isUInt or isInt depending on how it was
constructed. When you look at the current behavior, it really sound buggy...

I'm curious has existing code that would break with such change to the
semantic of isInt() and isUInt()...


>
> Perhaps the best way would have been to have users explicitly opt in
> for 64-bit support. That is, add an option to the constructor of
> Json::Reader that defaults to false, and if you set it to true then
> you're asserting that your code doesn't suffer from the problem
> mentioned above. I guess it's too late for that, though.
>

It is not too late. It should not be difficult to add a data member to
Features instance passed to the Reader to control its behavior that would
indicate if integer greater than 32 bits should be stored as double or int
in the Json::Value. Feel free to add this behavior, it should not be too
difficult.

That being said, when I look back at your code sample:
   unsigned int value =
       value.isUInt() ? value.asUInt() :
           (value.isDouble() ? value.asDouble() : kError);

I can not help but feel that something is wrong. If value does not fit on a
32 bits integer, then you will get mostly a random value after conversion to
double followed by a cast to unsigned int. Am I missing something?

Baptiste.

2011/5/23 Stephan Beal <sg...@go...>

> On Mon, May 23, 2011 at 1:50 PM, Baptiste Lepilleur <
> bap...@gm...> wrote:
>
>> I propose making the following changes:
>> 1) isUInt() returns true only asUInt() can succeed. Similar changes for
>> isInt().
>>     I think this contract should be generalized. e.g. asUInt() should
>> succeed if the value is a positive signed int Value.
>> 2) Adds new member functions:
>>    isUInt64() that returns true only if asUInt64() can succeed.
>>    and isInt64()...
>>
>> If I understood your problem correctly, this should solve your backward
>> compatibility issue. Is that correct?
>>
>
> My proposal would be to remove the int/uint distinction entirely, since
> JavaScript and JSON do not distinguish between the two. In my own JSON code
> i simply use int64_t for all integers and double for doubles (though JS
> there's just a single Number type, if i'm not mistaken). In my own use of
> json libs and JS engines (SpiderMonkey, QtScript, and Google v8), i've
> always found the distinction between signed/unsigned to be philosophically
> unsettling because JS doesn't natively have unsigned numbers.
>

In fact I think if we were just interested in supporting Javascript
compatibility we could probably get away with supporting only double.

There was an intermediate version of jsoncpp where the only integers type
were int64 / uint64 but this kind of change broke backward compatibility
painfully, for example causing printf( "%d", value.asInt()) to behave in
"interesting" way.

Also one of the aim of JsonCpp is for use for configuration file. In such
case you frequently need to deal with int/ unsigned int types. If the only
getter available were 64 bits integer we would be precision loss warning all
over the place.

Though if you use asUInt64() you can easily pretend that the other integer
type do not exist (minor the identified lack of isUInt64() with the right
semantic).


>  JSON, in fact, does not specify the precision of numeric values (
> http://www.ietf.org/rfc/rfc4627.txt?number=4627), probably because it
> would be impossible to guaranty that any given implementation (in umpteen[1]
> different programming languages) can honor that, e.g. a certain embedded
> environment might not have integers >32 bits. Section 2.4 of the JSON RFC
> notably does not specify any limits on numbers, so 2^75 (when written in
> expanded integer form) is syntactically legal, according to the grammar, but
> is almost certainly semantically illegal in any modern computer.
>
These pages:
>
> http://www.hunlock.com/blogs/The_Complete_Javascript_Number_Reference
>  http://www.jibbering.com/faq/#FAQ4_7
>
> say that in JS integers are reliable up to "15-16" digits (53 bits). Thus
> int64_t can legally hold any JS-legal value.
>
> Obviously, JSON is _not_ only for JavaScript, but i'm using JS as the
> baseline here because (A) JSON derives from JS and (B) it is primarily
> consumed by JS applications (though it is primarily generated by other
> technologies).
>
>
Actually, some parser leverage language features that bypass hardware
limitation. Python will happily parse a 2^75 integer. Support for use of the
arbitrary floating point precision type Decimal was even added at some point
in python. So there is clearly use cases around there where people just use
JSON for serialization and don't limit themselves to what can be done
portably.

Baptiste.


> [1] = American slang for "very many"
>
>

On Tue, May 24, 2011 at 3:06 AM, Baptiste Lepilleur
<bap...@gm...> wrote:
> To make sure I understand the issue: sscanf on Mac OS X will write in the
> format string provided as parameter even though it is declared const char *?

That's right. Don't ask me. :-/

> - JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
> multi-statements). IMHO it should just call a function that dump message and
> then call abort().

Sorry, I'm not sure I understand the first sentence here. Could you clarify?


> - partial duplication of assertions macros already present in json_value.cpp
>
> I think we should move the assertion/failure macros from json_value.cpp  to
> a common private header instead of duplicating them.
> In the future, we may want to provide hook to allow user to customize what
> happens on failure (e.g. like google breakpad integration or whatever stack
> capture tool you happen to use...)

Okay, sounds good.

On Tue, May 24, 2011 at 7:34 AM, Aaron Jacobs <ja...@go...> wrote:
> 2^64 - 1 -> isDouble

Sorry, this should be:

2^64 - 1 -> isUInt64, isDouble

On Mon, May 23, 2011 at 9:50 PM, Baptiste Lepilleur
<bap...@gm...> wrote:
> If I understood your problem correctly, this should solve your backward
> compatibility issue. Is that correct?

Almost but not quite, I think. Now the numbers in [2^32, 2^64) will be
parsed as uint64s, and the code above will always return kError
because both isUInt and isDouble return false, even though it used to
work for those values. I think maybe the only choice here is to have
each of the isFoo methods return true iff asFoo will work, whether
that's the best way to represent the number or not. For example:

0.7 -> isDouble
1 -> isUInt, isInt, isUInt64, isInt64, isDouble
2^32 -> isUInt, isUInt64, isDouble
2^40 -> isUInt64, isDouble
2^64 - 1 -> isDouble
2^64 -> isDouble

I can imagine that this change would cause problems for other existing
code, but I'm not sure. What do you think?

Perhaps the best way would have been to have users explicitly opt in
for 64-bit support. That is, add an option to the constructor of
Json::Reader that defaults to false, and if you set it to true then
you're asserting that your code doesn't suffer from the problem
mentioned above. I guess it's too late for that, though.

2011/5/23 Aaron Jacobs <ja...@go...>

> Hi Baptiste and jsoncpp-devel,
>
> I'm a software engineer at Google, where we use jsoncpp for a few projects.
> We've made a number of modifications internally, and I'd like to contribute
> them back to the public repository.
>
>
Thanks for contributing this back.

Below are some general comments concerning the individual patches.


> Here are a few patches that are ready to apply to SVN revision 192:
>
>    https://github.com/jacobsa/jsoncpp/commit/d7377e3c438490

=> OK

To make sure I understand the issue: sscanf on Mac OS X will write in the
format string provided as parameter even though it is declared const char *?

    https://github.com/jacobsa/jsoncpp/commit/7d2314b1fef8f7
=> OK

   https://github.com/jacobsa/jsoncpp/commit/1de4ede76d27b0

=> I see a few problems with this patch:
- JSON_FAIL_MESSAGE is bugged if there is no exception (e.g.
multi-statements). IMHO it should just call a function that dump message and
then call abort().
- partial duplication of assertions macros already present in json_value.cpp

I think we should move the assertion/failure macros from json_value.cpp  to
a common private header instead of duplicating them.

In the future, we may want to provide hook to allow user to customize what
happens on failure (e.g. like google breakpad integration or whatever stack
capture tool you happen to use...)

   https://github.com/jacobsa/jsoncpp/commit/8945bfb0a29235

=> OK

What's the easiest way to get these committed to the main jsoncpp SVN repo?
> I've got several more changes to contribute back, but wanted to get the
> ball
> rolling with these first.
>

I'll contact you privately.

Thanks for your time,
Baptiste.


> Thanks,
> Aaron Jacobs
>