Skip to content

Commit

Permalink
Merge pull request #1191 from RumbleDB/Version1.18
Browse files Browse the repository at this point in the history
Version1.18
  • Loading branch information
ghislainfourny authored Apr 12, 2022
2 parents 624b2a6 + 0940e3e commit 52a3424
Show file tree
Hide file tree
Showing 8 changed files with 23 additions and 23 deletions.
4 changes: 2 additions & 2 deletions docs/Docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Known issue

On occasion, the docker version of RumbleDB used to throw a Kryo NoSuchMethodError on some systems. This should be fixed with version 1.17, let us know if this is not the case.
On occasion, the docker version of RumbleDB used to throw a Kryo NoSuchMethodError on some systems. This should be fixed with version 1.18, let us know if this is not the case.

You can upgrade to the newest version with

Expand All @@ -29,7 +29,7 @@ The RumbleDB shell appears:
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.17.0 "Cacao Tree" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.18.0 "Cacao Tree" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/


Expand Down
8 changes: 4 additions & 4 deletions docs/HTTPServer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

RumbleDB can be run as an HTTP server that listens for queries. In order to do so, you can use the --server and --port parameters:

spark-submit rumbledb-1.17.0.jar serve -p 8001
spark-submit rumbledb-1.18.0.jar serve -p 8001

This command will not return until you force it to (Ctrl+C on Linux and Mac). This is because the server has to run permanently to listen to incoming requests.

Expand Down Expand Up @@ -94,19 +94,19 @@ Then there are two options
- Connect to the master with SSH with an extra parameter for securely tunneling the HTTP connection (for example `-L 8001:localhost:8001` or any port of your choosing)
- Download the RumbleDB jar to the master node

wget https://github.com/RumbleDB/rumble/releases/download/v1.12.0/rumbledb-1.17.0.jar
wget https://github.com/RumbleDB/rumble/releases/download/v1.12.0/rumbledb-1.18.0.jar

- Launch the HTTP server on the master node (it will be accessible under `http://localhost:8001/jsoniq`).

spark-submit rumbledb-1.17.0.jar serve -p 8001
spark-submit rumbledb-1.18.0.jar serve -p 8001

- And then use Jupyter notebooks in the same way you would do it locally (it magically works because of the tunneling)

### With the EC2 hostname

There is also another way that does not need any tunnelling: you can specify the hostname of your EC2 machine (copied over from the EC2 dashboard) with the --host parameter. For example, with the placeholder <ec2-hostname>:

spark-submit rumbledb-1.17.0.jar serve -p 8001 -h <ec2-hostname>
spark-submit rumbledb-1.18.0.jar serve -p 8001 -h <ec2-hostname>

You also need to make sure in your EMR security group that the chosen port (e.g., 8001) is accessible from the machine in which you run your Jupyter notebook. Then, you can point your Jupyter notebook on this machine to `http://<ec2-hostname>:8001/jsoniq`.

Expand Down
2 changes: 1 addition & 1 deletion docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ After successful completion, you can check the `target` directory, which should

The most straightforward to test if the above steps were successful is to run the RumbleDB shell locally, like so:

$ spark-submit target/rumbledb-1.17.0.jar repl
$ spark-submit target/rumbledb-1.18.0.jar repl

The RumbleDB shell should start:

Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
site_name: RumbleDB 1.17 "Cacao Tree" beta
site_name: RumbleDB 1.18 "Scarlet Ixora" beta
pages:
- '1. Documentation home': 'index.md'
- '2. Getting started': 'Getting started.md'
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/assets/banner.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.17.0 "Cacao Tree" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.18.0 "Scarlet Ixora" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/

24 changes: 12 additions & 12 deletions src/main/resources/assets/defaultscreen.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,46 +17,46 @@ for directly running a query from an input file or (with -q) provided directly o

It is the default mode.

spark-submit rumbledb-1.17.0.jar run my-query.jq
spark-submit rumbledb-1.17.0.jar run -q '1+1'
spark-submit rumbledb-1.18.0.jar run my-query.jq
spark-submit rumbledb-1.18.0.jar run -q '1+1'

You can specify an output path with -o like so:
spark-submit rumbledb-1.17.0.jar run -q '1+1' -o my-output.txt
spark-submit rumbledb-1.18.0.jar run -q '1+1' -o my-output.txt

**** serve ****
for running as an HTTP server listening on the specified port (-p) and host (-h).

spark-submit rumbledb-1.17.0.jar serve -p 9090
spark-submit rumbledb-1.18.0.jar serve -p 9090

RumbleDB also supports Apache Livy for use in Jupyter notebooks, which may be
even more convenient if you are using a cluster.

**** repl ****
for shell mode.

spark-submit rumbledb-1.17.0.jar repl
spark-submit rumbledb-1.18.0.jar repl


**** resource use configuration ****

For a local use, you can control the number of cores, as well as allocated
memory, with:
spark-submit --master local[*] rumbledb-1.17.0.jar repl
spark-submit --master local[*] rumbledb-1.17.0.jar repl
spark-submit --master local[2] rumbledb-1.17.0.jar repl
spark-submit --master local[*] --driver-memory 10G rumbledb-1.17.0.jar repl
spark-submit --master local[*] rumbledb-1.18.0.jar repl
spark-submit --master local[*] rumbledb-1.18.0.jar repl
spark-submit --master local[2] rumbledb-1.18.0.jar repl
spark-submit --master local[*] --driver-memory 10G rumbledb-1.18.0.jar repl

You can use RumbleDB remotely with:
spark-submit --master yarn rumbledb-1.17.0.jar repl
spark-submit --master yarn rumbledb-1.18.0.jar repl

(Although for clusters provided as a service, --master yarn is often implicit
and unnecessary).

For remote use (e.g., logged in on the Spark cluster with ssh), you can set the
number of executors, cores and memory, you can use:
spark-submit --executor-cores 3 --executor-memory 5G rumbledb-1.17.0.jar repl
spark-submit --executor-cores 3 --executor-memory 5G rumbledb-1.18.0.jar repl

For remote use, you can also use other file system paths such as S3, HDFS, etc:
spark-submit rumbledb-1.17.0.jar run hdfs://server:port/my-query.jq -o hdfs://server:port/my-output.json
spark-submit rumbledb-1.18.0.jar run hdfs://server:port/my-query.jq -o hdfs://server:port/my-output.json

More documentation on available CLI parameters is available on https://www.rumbledb.org/
2 changes: 1 addition & 1 deletion src/main/resources/assets/jsound-validator.html
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ <h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif
</span>
</div>
<div>
<button id="submit" style="margin: 0 auto; padding: 10px;">Validate with RumbleDB 1.17.0</button>
<button id="submit" style="margin: 0 auto; padding: 10px;">Validate with RumbleDB 1.18.0</button>
</div>

<h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif; color: #336699;">Results</h2>
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/assets/public.html
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ <h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif
</span>
</div>
<div>
<button id="submit" style="margin: 20px; padding: 10px;">Evaluate with RumbleDB 1.17.0</button>
<button id="submit" style="margin: 20px; padding: 10px;">Evaluate with RumbleDB 1.18.0</button>
</div>

</div>
Expand Down

0 comments on commit 52a3424

Please sign in to comment.