Skip to content
/ gpjson Public
forked from koesie10/gpjson

GPU-based JSON data processing system accessible via all GraalVM languages

Notifications You must be signed in to change notification settings

necst/gpjson

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GpJSON - Leveraging Structural Indexes for High-Performance JSON Data Processing on GPUs

This Truffle language exposes JSONPath query execution to the polyglot GraalVM.

The goals are:

  1. Present a couple of parsing techniques based on structural indexes to quickly execute queries on JSON files
  2. Introduce a batching approach to improve performance and allow the processing of datasets bigger than the GPU’s memory
  3. Implement the above concepts into a Truffle Language to provide an engine that can be used from any host language that can run on the GraalVM.

Using GpJSON in the GraalVM

To compile a JAR file containing GpJSON move to the language folder and run mvn package.

Next, copy the JAR file from target/gpjson.jar into jre/languages/gpjson (Java 8) or languages/gpjson (Java 11) of the Graal installation.

Note that --jvm and --polyglot must be specified in both cases as well.

In the examples folder, you can find a couple of files containing examples of the GpJSON's syntax.

Benchmarks suite

To run the benchmarks provided in the benchmarks folder you first need to install the following dependencies:

Then, add the following variables to your .bashrc (or equivalent):

export CUDA_DIR=[your-cuda-path]
export PATH=$PATH:$CUDA_DIR/bin
export GRAAL_DIR=[your-graalvm-path]
export PATH=$PATH:$GRAAL_DIR/bin
export NODE_DIR=[your-node-path]

Copy the grcuda and gpjson JARs from the deliverables folder to [your-graalvm-path]/languages/[grcuda/gpjson]/.

Move to the benchmarks folder cd benchmarks and run make setup to install jsonpath, jsonpath-plus and simdjson.

Finally, run ./[name-of-the-benchmark].sh. Results will be saved to [name-of-the-benchmark].csv.

The following options can be added to the command above:

  • -g to exclude the GPU-based benchmarks (GpJSON only). Default is false
  • -w [number] to set the number of warmup runs. Default is 5
  • -r [number] to set the number of runs. Default is 10
  • -t [number] to set the number of threads (Java JSONPath only). Default is 11
  • -d [path] to set the path of the dataset. Default value is /home/ubuntu/datasets-ext/

Datasets can be downloaded here.

For further details, such as the versions of the dependencies used or the queries executed by the benchmarks suite, please refer to the official thesis and/or publication.

About

GPU-based JSON data processing system accessible via all GraalVM languages

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 90.1%
  • Java 4.5%
  • Python 1.9%
  • JavaScript 1.2%
  • Cuda 0.9%
  • C 0.8%
  • Other 0.6%