summaryrefslogtreecommitdiff
path: root/src/backend/utils/adt/jsonfuncs.c
AgeCommit message (Collapse)Author
2017-06-21Phase 3 of pgindent updates.Tom Lane
Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected]
2017-06-21Phase 2 of pgindent updates.Tom Lane
Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected]
2017-06-21Initial pgindent run with pg_bsd_indent version 2.0.Tom Lane
The new indent version includes numerous fixes thanks to Piotr Stefaniak. The main changes visible in this commit are: * Nicer formatting of function-pointer declarations. * No longer unexpectedly removes spaces in expressions using casts, sizeof, or offsetof. * No longer wants to add a space in "struct structname *varname", as well as some similar cases for const- or volatile-qualified pointers. * Declarations using PG_USED_FOR_ASSERTS_ONLY are formatted more nicely. * Fixes bug where comments following declarations were sometimes placed with no space separating them from the code. * Fixes some odd decisions for comments following case labels. * Fixes some cases where comments following code were indented to less than the expected column 33. On the less good side, it now tends to put more whitespace around typedef names that are not listed in typedefs.list. This might encourage us to put more effort into typedef name collection; it's not really a bug in indent itself. There are more changes coming after this round, having to do with comment indentation and alignment of lines appearing within parentheses. I wanted to limit the size of the diffs to something that could be reviewed without one's eyes completely glazing over, so it seemed better to split up the changes as much as practical. Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected]
2017-05-29Make edge-case behavior of jsonb_populate_record match json_populate_recordTom Lane
json_populate_record throws an error if asked to convert a JSON scalar or array into a composite type. jsonb_populate_record was returning a record full of NULL fields instead. It seems better to make it throw an error for this case as well. Nikita Glukhov Discussion: https://postgr.es/m/[email protected]
2017-05-29Fix thinko in JsObjectSize() macro.Tom Lane
The macro gave the wrong answers for a JsObject with is_json == 0: it would return 1 if jsonb_cont == NULL, or if that wasn't NULL, it would return 1 for any non-zero size. We could fix that, but the only use of this macro at present is in the JsObjectIsEmpty() macro, so it seems simpler and clearer to get rid of JsObjectSize() and put corrected logic into JsObjectIsEmpty(). Thinko in commit cf35346e8, so no need for back-patch. Nikita Glukhov Discussion: https://postgr.es/m/[email protected]
2017-05-19Fix misspelled struct tag.Tom Lane
This was evidently intended to match the struct's typedef name, but it didn't quite. Noted while testing find_typedefs.
2017-05-19Fix argument name differencesPeter Eisentraut
Different names were used between function declaration and definition.
2017-05-17Post-PG 10 beta1 pgindent runBruce Momjian
perltidy run not included.
2017-04-07Remove extraneous comma to satisfy picky compilerAndrew Dunstan
per buildfarm
2017-04-07Make json_populate_record and friends operate recursivelyAndrew Dunstan
With this change array fields are populated from json(b) arrays, and composite fields are populated from json(b) objects. Along the way, some significant code refactoring is done to remove redundancy in the way to populate_record[_set] and to_record[_set] functions operate, and some significant efficiency gains are made by caching tuple descriptors. Nikita Glukhov, edited some by me. Reviewed by Aleksander Alekseev and Tom Lane.
2017-03-31Transform or iterate over json(b) string valuesAndrew Dunstan
Dmitry Dolgov, reviewed and lightly edited by me.
2017-03-12Use wrappers of PG_DETOAST_DATUM_PACKED() more.Noah Misch
This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-12Assume deconstruct_array() outputs are untoasted.Noah Misch
In functions that issue a deconstruct_array() call, consistently use plain VARSIZE()/VARDATA() on the array elements. Prior practice was divided between those and VARSIZE_ANY_EXHDR()/VARDATA_ANY().
2017-01-25Introduce convenience macros to hide JsonbContainer header accesses better.Tom Lane
This improves readability a bit and may make future improvements easier. In passing, make sure that the JB_ROOT_IS_XXX macros deliver boolean (0/1) results; the previous coding was a bug hazard, though no actual bugs are known. Nikita Glukhov, extended a bit by me Discussion: https://postgr.es/m/[email protected]
2017-01-22Relocate static function declarations to be after typedefs in jsonfuncs.c.Tom Lane
Project style is to put things in this order, for the good and sufficient reason that you often need the typedefs in the function declarations. There already was one function declaration that needed a typedef, which was randomly placed away from all the other static function declarations in consequence. And the submitted patch for better json_populate_record functionality jumped through even more hoops in order to preserve this bad idea. This patch only moves lines from point A to point B, no other changes.
2017-01-18Implement array version of jsonb_delete and operatorMagnus Hagander
This makes it possible to delete multiple keys from a jsonb value by passing in an array of text values, which makes the operaiton much faster than individually deleting the keys (which would require copying the jsonb structure over and over again. Reviewed by Dmitry Dolgov and Michael Paquier
2017-01-03Update copyright via script for 2017Bruce Momjian
2016-11-14Fix typo in commentMagnus Hagander
The function was renamed in 908e23473, but the comment never learned about it.
2016-10-13Fix broken jsonb_set() logic for replacing array elements.Tom Lane
Commit 0b62fd036 did a fairly sloppy job of refactoring setPath() to support jsonb_insert() along with jsonb_set(). In its defense, though, there was no regression test case exercising the case of replacing an existing element in a jsonb array. Per bug #14366 from Peng Sun. Back-patch to 9.6 where bug was introduced. Report: <[email protected]>
2016-08-27Add macros to make AllocSetContextCreate() calls simpler and safer.Tom Lane
I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <[email protected]>
2016-07-26Fix typoPeter Eisentraut
2016-06-09pgindent run for 9.6Robert Haas
2016-04-06Add jsonb_insertTeodor Sigaev
It inserts a new value into an jsonb array at arbitrary position or a new key to jsonb object. Author: Dmitry Dolgov Reviewers: Petr Jelinek, Vitaly Burovoy, Andrew Dunstan
2016-03-23Code review for error reports in jsonb_set().Tom Lane
User-facing (even tested by regression tests) error conditions were thrown with elog(), hence had wrong SQLSTATE and were untranslatable. And the error message texts weren't up to project style, either.
2016-03-23Fix unsafe use of strtol() on a non-null-terminated Text datum.Tom Lane
jsonb_set() could produce wrong answers or incorrect error reports, or in the worst case even crash, when trying to convert a path-array element into an integer for use as an array subscript. Per report from Vitaly Burovoy. Back-patch to 9.5 where the faulty code was introduced (in commit c6947010ceb42143). Michael Paquier
2016-03-03Fix json_to_record() bug with nested objects.Tom Lane
A thinko concerning nesting depth caused json_to_record() to produce bogus output if a field of its input object contained a sub-object with a field name matching one of the requested output column names. Per bug #13996 from Johann Visagie. I added a regression test case based on his example, plus parallel tests for json_to_recordset, jsonb_to_record, jsonb_to_recordset. The latter three do not exhibit the same bug (which suggests that we may be missing some opportunities to share code...) but testing seems like a good idea in any case. Back-patch to 9.4 where these functions were introduced.
2016-01-02Update copyright for 2016Bruce Momjian
Backpatch certain files through 9.1
2015-12-11Improve some messagesPeter Eisentraut
2015-10-12Use JsonbIteratorToken consistently in automatic variable declarations.Noah Misch
Many functions stored JsonbIteratorToken values in variables of other integer types. Also, standardize order relative to other declarations. Expect compilers to generate the same code before and after this change.
2015-10-05Prevent stack overflow in json-related functions.Noah Misch
Sufficiently-deep recursion heretofore elicited a SIGSEGV. If an application constructs PostgreSQL json or jsonb values from arbitrary user input, application users could have exploited this to terminate all active database connections. That applies to 9.3, where the json parser adopted recursive descent, and later versions. Only row_to_json() and array_to_json() were at risk in 9.2, both in a non-security capacity. Back-patch to 9.2, where the json type was introduced. Oskari Saarenmaa, reviewed by Michael Paquier. Security: CVE-2015-5289
2015-10-04Disallow invalid path elements in jsonb_setAndrew Dunstan
Null path elements and, where the object is an array, invalid integer elements now cause an error. Incorrect behaviour noted by Thom Brown, patch from Dmitry Dolgov. Backpatch to 9.5 where jsonb_set was introduced
2015-09-13Fix the fastpath rule for jsonb_concat with an empty operand.Andrew Dunstan
To prevent perverse results, we now only return the other operand if it's not scalar, and if both operands are of the same kind (array or object). Original bug complaint and patch from Oskari Saarenmaa, extended by me to cover the cases of different kinds of jsonb. Backpatch to 9.5 where jsonb_concat was introduced.
2015-07-28Only adjust negative indexes in json_get up to the length of the path.Andrew Dunstan
The previous code resulted in memory access beyond the path bounds. The cure is to move it into a code branch that checks the value of lex_level is within the correct bounds. Bug reported and diagnosed by Piotr Stefaniak.
2015-07-19Remove dead code.Andrew Dunstan
Defect noticed by Coverity.
2015-07-18Support JSON negative array subscripts everywhereAndrew Dunstan
Previously, there was an inconsistency across json/jsonb operators that operate on datums containing JSON arrays -- only some operators supported negative array count-from-the-end subscripting. Specifically, only a new-to-9.5 jsonb deletion operator had support (the new "jsonb - integer" operator). This inconsistency seemed likely to be counter-intuitive to users. To fix, allow all places where the user can supply an integer subscript to accept a negative subscript value, including path-orientated operators and functions, as well as other extraction operators. This will need to be called out as an incompatibility in the 9.5 release notes, since it's possible that users are relying on certain established extraction operators changed here yielding NULL in the event of a negative subscript. For the json type, this requires adding a way of cheaply getting the total JSON array element count ahead of time when parsing arrays with a negative subscript involved, necessitating an ad-hoc lex and parse. This is followed by a "conversion" from a negative subscript to its equivalent positive-wise value using the count. From there on, it's as if a positive-wise value was originally provided. Note that there is still a minor inconsistency here across jsonb deletion operators. Unlike the aforementioned new "-" deletion operator that accepts an integer on its right hand side, the new "#-" path orientated deletion variant does not throw an error when it appears like an array subscript (input that could be recognized by as an integer literal) is being used on an object, which is wrong-headed. The reason for not being stricter is that it could be the case that an object pair happens to have a key value that looks like an integer; in general, these two possibilities are impossible to differentiate with rhs path text[] argument elements. However, we still don't allow the "#-" path-orientated deletion operator to perform array-style subscripting. Rather, we just return the original left operand value in the event of a negative subscript (which seems analogous to how the established "jsonb/json #> text[]" path-orientated operator may yield NULL in the event of an invalid subscript). In passing, make SetArrayPath() stricter about not accepting cases where there is trailing non-numeric garbage bytes rather than a clean NUL byte. This means, for example, that strings like "10e10" are now not accepted as an array subscript of 10 by some new-to-9.5 path-orientated jsonb operators (e.g. the new #- operator). Finally, remove dead code for jsonb subscript deletion; arguably, this should have been done in commit b81c7b409. Peter Geoghegan and Andrew Dunstan
2015-06-12Fix "path" infrastructure bug affecting jsonb_set()Andrew Dunstan
jsonb_set() and other clients of the setPathArray() utility function could get spurious results when an array integer subscript is provided that is not within the range of int. To fix, ensure that the value returned by strtol() within setPathArray() is within the range of int; when it isn't, assume an invalid input in line with existing, similar cases. The path-orientated operators that appeared in PostgreSQL 9.3 and 9.4 do not call setPathArray(), and already independently take this precaution, so no change there. Peter Geoghegan
2015-06-08Desupport jsonb subscript deletion on objectsAndrew Dunstan
Supporting deletion of JSON pairs within jsonb objects using an array-style integer subscript allowed for surprising outcomes. This was mostly due to the implementation-defined ordering of pairs within objects for jsonb. It also seems desirable to make jsonb integer subscript deletion consistent with the 9.4 era general purpose integer subscripting operator for jsonb (although that operator returns NULL when an object is encountered, while we prefer here to throw an error). Peter Geoghegan, following discussion on -hackers.
2015-06-01Avoid naming a variable "new", and remove bogus initializer.Andrew Dunstan
Per gripe from Tom Lane.
2015-06-01Add a couple of missing JsonbValue type initialisers.Andrew Dunstan
2015-06-01Rename jsonb_replace to jsonb_set and allow it to add new valuesAndrew Dunstan
The function is given a fourth parameter, which defaults to true. When this parameter is true, if the last element of the path is missing in the original json, jsonb_set creates it in the result and assigns it the new value. If it is false then the function does nothing unless all elements of the path are present, including the last. Based on some original code from Dmitry Dolgov, heavily modified by me. Catalog version bumped.
2015-05-27Revert "Simplify addJsonbToParseState()"Andrew Dunstan
This reverts commit fba12c8c6c4159e1923958a4006b26f3cf873254. This relied on a commit that is also being reverted.
2015-05-26Simplify addJsonbToParseState()Andrew Dunstan
This function no longer needs to walk non-scalar structures passed to it, following commit 54547bd87f49326d67051254c363e6597d16ffda.
2015-05-25Clean up and simplify jsonb_concat code.Andrew Dunstan
Some of this is made possible by commit 9b74f32cdbff8b9be47fc69164eae552050509ff which lets pushJsonbValue handle binary Jsonb values, meaning that clients no longer have to, and some is just doing things in simpler and more straightforward ways.
2015-05-24pgindent run for 9.5Bruce Momjian
2015-05-17Fix typos in commentsMagnus Hagander
Dmitriy Olshevskiy
2015-05-13Fix jsonb replace and delete on scalars and empty structuresAndrew Dunstan
These operations now error out if attempted on scalars, and simply return the input if attempted on empty arrays or objects. Along the way we remove the unnecessary cloning of the input when it's known to be unchanged. Regression tests covering these cases are added.
2015-05-12Additional functions and operators for jsonbAndrew Dunstan
jsonb_pretty(jsonb) produces nicely indented json output. jsonb || jsonb concatenates two jsonb values. jsonb - text removes a key and its associated value from the json jsonb - int removes the designated array element jsonb - text[] removes a key and associated value or array element at the designated path jsonb_replace(jsonb,text[],jsonb) replaces the array element designated by the path or the value associated with the key designated by the path with the given value. Original work by Dmitry Dolgov, adapted and reworked for PostgreSQL core by Andrew Dunstan, reviewed and tidied up by Petr Jelinek.
2015-05-04Fix two small bugs in json's populate_record_workerAndrew Dunstan
The first bug is not releasing a tupdesc when doing an early return out of the function. The second bug is a logic error in choosing when to do an early return if given an empty jsonb object. Bug reports from Pavel Stehule and Tom Lane respectively. Backpatch to 9.4 where these were introduced.
2015-03-31Remove spurious semicolons.Heikki Linnakangas
Petr Jelinek
2015-02-20Use FLEXIBLE_ARRAY_MEMBER in struct RecordIOData.Tom Lane
I (tgl) fixed this last night in rowtypes.c, but I missed that the code had been copied into a couple of other places. Michael Paquier