Nebula faces this issue too, in this short note, I would just like to summarize all places with the concern and how we handle them for now, this doesn't mean it is ideal, in fact, we are still looking for improvements.
We have been using RapidJson as the JSON serde library in our C++ runtime, due to its capability handling this concern well.
We don't do
long<-->string conversion trick. Hence we removed similar trick in your code base, specifically in InExpression serde code path.
node.js is the web server runtime
Nebula Web resides in, it is definitely the most hot place regarding this issue. We have been using JSON as data fromat to exchange requests/responses between
web client and
Nebula Server. Here are what we have done:
- We're using json-bigint package to do
json.parseso if the input JSON includes bigint values, it won't lose the value're precision.
- In nebula proto, we added two more types of predicate, so it allows client to use one of them
- value: string type
- nvalue: int64 type
- dvalue: double type.
with this change, we don't pay string->int or string->double type data conversion. (In fact, protobuf does this for us - it is uncertain if perf is better, but at least cleaner at interface).
The serde work is dene by protobuf-js, which is still using
Number for all number values, so the problem still exists, if passing bigint value in
grpc call defined by int64 in
nebula.proto, Nebula Server may receive wrong value. And I haven't seen this is going to be fixed soon anywhere. We will keep eyes open for any alternative solution.
As a work-around, we will use "string" representation for
nvalue by appending
[jstype = JS_STRING] in its proto declaration. After that, we requires client to send string list for this field for now. Definitely this is neither convinient nor efficient.
Nebula UI which means browser here, we haven't handle any thing like this, since it is for display purpose, basically all bigint values are returned in
string type. Client code (browser js) needs to convert it if used in further computation.
curl to post a JSON blob of bigint array,
Nebula API (node.js) won't lose precision of values. Other clients like Java should be fine too.