Types to interoperate with applications that make full use of JSON.
und expects no breaking change except for MarshalJSONV2
and UnmarshalJSONV2
methods.
Normally those methods are not used directly by users, so this expected breakage should not incur visible effects for most of them.
- und waits for release of
encoding/json/v2
- und depends on
github.com/go-json-experiment/json
which is an experimentalencoding/json/v2
implementation. - Types defined in this module implement
json.MarshalerV2
andjson.UnmarshalerV2
. - Eventually the dependency would be swapped to
encoding/json/v2
and those methods would be changed to usev2
, - or erased completely if
v2
decides not to continue to use that names.
- und depends on
run example by go run github.com/ngicks/und/[email protected]
.
You'll see zero value fields whose type is defined under this module are omitted by jsonv2(github.com/go-json-experiment/json
) with omitzero
json option.
Also types defined under sliceund
and sliceund/elastic
are omitted by encoding/json
v1 if zero, with omitempty
struct tag option.
package main
import (
"encoding/json"
"fmt"
"github.com/ngicks/und"
"github.com/ngicks/und/elastic"
"github.com/ngicks/und/option"
jsonv2 "github.com/go-json-experiment/json"
"github.com/go-json-experiment/json/jsontext"
"github.com/ngicks/und/sliceund"
sliceelastic "github.com/ngicks/und/sliceund/elastic"
)
type sample1 struct {
Foo string
Bar und.Und[nested1] `json:",omitzero"`
Baz elastic.Elastic[nested1] `json:",omitzero"`
Qux sliceund.Und[nested1] `json:",omitzero"`
Quux sliceelastic.Elastic[nested1] `json:",omitzero"`
}
type nested1 struct {
Bar und.Und[string] `json:",omitzero"`
Baz elastic.Elastic[int] `json:",omitzero"`
Qux sliceund.Und[float64] `json:",omitzero"`
Quux sliceelastic.Elastic[bool] `json:",omitzero"`
}
type sample2 struct {
Foo string
Bar und.Und[nested2] `json:",omitempty"`
Baz elastic.Elastic[nested2] `json:",omitempty"`
Qux sliceund.Und[nested2] `json:",omitempty"`
Quux sliceelastic.Elastic[nested2] `json:",omitempty"`
}
type nested2 struct {
Bar und.Und[string] `json:",omitempty"`
Baz elastic.Elastic[int] `json:",omitempty"`
Qux sliceund.Und[float64] `json:",omitempty"`
Quux sliceelastic.Elastic[bool] `json:",omitempty"`
}
func main() {
s1 := sample1{
Foo: "foo",
Bar: und.Defined(nested1{Bar: und.Defined("foo")}),
Baz: elastic.FromValue(nested1{Baz: elastic.FromOptions([]option.Option[int]{option.Some(5), option.None[int](), option.Some(67)})}),
Qux: sliceund.Defined(nested1{Qux: sliceund.Defined(float64(1.223))}),
Quux: sliceelastic.FromValue(nested1{Quux: sliceelastic.FromOptions([]option.Option[bool]{option.None[bool](), option.Some(true), option.Some(false)})}),
}
var (
bin []byte
err error
)
bin, err = jsonv2.Marshal(s1, jsontext.WithIndent(" "))
if err != nil {
panic(err)
}
fmt.Printf("marshaled by v2=\n%s\n", bin)
// see? undefined (=zero value) fields are omitted.
/*
marshaled by v2=
{
"Foo": "foo",
"Bar": {
"Bar": "foo"
},
"Baz": [
{
"Baz": [
5,
null,
67
]
}
],
"Qux": {
"Qux": 1.223
},
"Quux": [
{
"Quux": [
null,
true,
false
]
}
]
}
*/
s2 := sample2{
Foo: "foo",
Bar: und.Defined(nested2{Bar: und.Defined("foo")}),
Baz: elastic.FromValue(nested2{Baz: elastic.FromOptions([]option.Option[int]{option.Some(5), option.None[int](), option.Some(67)})}),
Qux: sliceund.Defined(nested2{Qux: sliceund.Defined(float64(1.223))}),
Quux: sliceelastic.FromValue(nested2{Quux: sliceelastic.FromOptions([]option.Option[bool]{option.None[bool](), option.Some(true), option.Some(false)})}),
}
bin, err = json.MarshalIndent(s2, "", " ")
if err != nil {
panic(err)
}
fmt.Printf("marshaled by v1=\n%s\n", bin)
// You see. Types defined under ./sliceund/ can be omitted by encoding/json.
// Types defined in ./ and ./elastic cannot be omitted by it.
/*
marshaled by v1=
{
"Foo": "foo",
"Bar": {
"Bar": "foo",
"Baz": null
},
"Baz": [
{
"Bar": null,
"Baz": [
5,
null,
67
]
}
],
"Qux": {
"Bar": null,
"Baz": null,
"Qux": 1.223
},
"Quux": [
{
"Bar": null,
"Baz": null,
"Quux": [
null,
true,
false
]
}
]
}
*/
}
When processing JSON values in GO, normally, at least I assume, you define a type that matches schema of JSON value, and use them with encoding/json
.
(Of course there are numbers of third party modules that process JSON nicely, but I place them out of scope.)
I think you'll normally specify *T
as field type to allow it to be empty. This treats undefined and null fields equally (unless you use non-zero value for an unmarshale target.)
This works fine in many cases. However sometimes its simplicity conflicts the concept of JSON.
As you can see in Open API spec,
JSON naturally has concept of absent fields(field is not specified in required
section),
and also nullable field(nullable
attribute is set to true
)
The difference of null and undefined does matter in some common practice.
For example, Elasticsearch allows users to send partial document JSON to Update part of a document. The Elasticsearch treats all of undefined
(absent), null
, []
equally; nonexistent field. So the partial update API skips updating of undefined fields and clears the fields if corresponding field of input JSON is null
.
How do you achieve this partial update in Go?
I suspect simplest and most straightforward way is using map[string]any
as a staging data.
If a field type implements json.Unmarshaler
, encoding/json
calls this method while unamrashaling incoming JSON value only if there's matching field in the data, even when the value is null
literal.
Therefore, T | null | undefined
can be easily mapped from 3 state where: UnmarshalJSON
was called with non-null data | was called with null
literal | was not called, respectively.
The problem arises when marshaling the struct if the field has distinct T | null | undefined
state; encoding/json
does not omit struct. This is why you end up always specifying *time.Time
as field type instead of time.Time
, if you want to omit zero value of the time.
Most simplest way to omit zero struct, I think, is using map[string]any
as staging data.
You can use github.com/jeevatkm/go-model to map any arbitrary structs into map[string]any
. Then you can remove any arbitrary field from that. (You can't use the popular mapstructure to achieve this because of #334). Finally you can marshal map[string]any
via json.Marshal
.
This should incur unnecessary overhead to marshaling, also feels clumsier and tiring.
As you can see in here: https://github.com/golang/go/blob/go1.22.5/src/encoding/json/encode.go#L306-L318 ,
omitempty
works on []T
and map[K]V
.
With generics introduced in Go1.18, you can define a []T
based type with convenient methods. The only drawback is that you can't hide internal data structure for those type. Any change to that should be considered a breaking change. But it's ok because I suspect there's not a lot of chance of changing around it.
I've defined type like type Option[T any]{valid bool; t T}
to express some or none. Then I combine this with []T
to express T | null | undefined
so that I can limit length of slice to 1. The capacity for the slice always stays 1, allocated only once and no chance of growth afterwards, no excess memory consumption(only a single bool
flag field).
As a conclusion, this module defines []Option[T]
based types to handle T | null | undefined
easily.
Option[T]
: Rust-like optional value.- can be Some or None.
- is comparable if
T
is comparable. - have
Equal
method in caseT
is not comparable or comparable but needs custom equality tests(e.g.time.Time
) - has convenient methods stolen from rust's
option<T>
- can be used in place of
*T
- is copied by assign.
Other types are based on Option[T]
.
Und[T]
: undefined, null orT
Elastic[T]
: undefined, null,T
or [](T
| null)- mainly for consuming elasticsearch JSON documents.
- or configuration files.
There are 2 variants
github.com/ngicks/und
:Option[Option[T]]
based types.- omitted only if encoding through
github.com/go-json-experiment/json
(possibly a futureencoding/json/v2
) with the,omitzero
options- jsoniter with
,omitempty
and custom encoder.
- omitted only if encoding through
github.com/ngicks/und/sliceund
:[]Option[T]
based types.- omitted with
,omitempty
.
- omitted with