One Sketch Away

Json Documents Using sql And Soda

Created by Ranjith Vijayan on Aug 17, 2022

This article covers an example of how to easily access JSON documents in Oracle DB without loading! This approach may be useful for some integration use-cases wherein json data needs to be processed/uploaded into traditional tables. Instead of writing complicated parsing & loading routines, the “schema-less” datamodel using JSON will be a natural fit for RESTful development techniques, and the power of SQL would come in handy for analytic requirements.

Oracle 21c introduced a new JSON data type. You should use this in preference to other data types. More details here.

The JSON document file. The example given in this doc is based on Oracle DB JSON Developer’s Guide. A sample file is given in Github that is a typical json dump format of NOSQL databases. i.e. its JSON objects separated by CRLF The use-case I was working on was for ingesting data from external database (MSSQL) - The queries used to produce JSON output in desired format is given here for reference. We use a docker container of mssql for this example.

docker run --name mssql -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=Y0urStr0ngPassw0rd" -p 1433:1433 -v /scratch/db-docker/obpm-db-14-6/share/:/share -d mcr.microsoft.com/mssql/server:2019-latest
docker exec -it mssql bash -c "/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P 'Y0urStr0ngPassw0rd' -Q 'set nocount on;SELECT [value] FROM OPENJSON((select * from sys.databases for json PATH))' -y0 -o /share/data.json"
    The above query using sqlcmd will output the table sys.databases into data.json with following content. query with "for JSON PATH" produces the output in JSON format. -y0 is for not truncating the data. "set nocount on" is basically equivalent of "set feed off" in sql*plus.

Lets "load" this in Oracle without loading. The file we created is in a directory that Oracle DB can access (if not move this file to make it available for Oracle database)
        CREATE TABLE json_dump_file_contents (json_document BLOB)
          ORGANIZATION EXTERNAL (TYPE ORACLE_LOADER DEFAULT DIRECTORY JSON_DATA
                                 ACCESS PARAMETERS
                                   (RECORDS DELIMITED BY 0x'0A'
                                    DISABLE_DIRECTORY_LINK_CHECK
                                    FIELDS (json_document CHAR(32000)))
                                 LOCATION (JSON_DATA:'data.json'))
          PARALLEL
          REJECT LIMIT UNLIMITED;

```