Documentation

Important References

IMPORTANT: You will need the service_account.json, the API key, and the URL for the database sent in your purchase confirmation email.

The complete API is defined in the proto file found under the gRPC code examples. These are labeled by rpc in the RogueDB service. All responses are the Response message. All inputs are captured in the parentheses (Insert, Update, Remove, Search).

‍

Complete Guide

The code examples covers authentication, all 4 CRUD APIs, and schema change with detailed notes on usage. Included are functions for JWT token generation used for authentication and file detection.

‍

Learn how to use REST and JSON with RogueDB.

Python
C++
Go
import datetime
import json
import jwt
import time
import os
from pathlib import Path

# REST and JSON Imports
import requests

# External Dependency Requirements:
# PyJWT, requests

def create_jwt(service_account: str, expire_minutes: int = 60) -> str:
    # NOTE: Replace with file path to service_account.json
    # from purchase confirmation email.
    with open(service_account) as input_file:
        key_data = json.load(input_file)

    now = datetime.datetime.now(datetime.timezone.utc)
    iat = int(time.mktime(now.timetuple()))
    exp = int(time.mktime((now + datetime.timedelta(minutes=expire_minutes)).timetuple()))

    payload = {
        "iat": iat,
        "exp": exp,
        "iss": key_data['client_email'],
        "aud": f"{key_data['client_email'].split('@')[0]}.roguedb.dev", 
        "sub": key_data['client_email'] }

    headers = {
        "kid": key_data['private_key_id'],
        "alg": "RS256", # Google Cloud service accounts typically use RS256
        "typ": "JWT" }

    return jwt.encode(
        payload, key_data['private_key'], 
        algorithm="RS256", headers=headers)

def detect_files(directories: list[str]) -> list[Path]:
    files = []
    for directory in directories:
        with os.scandir(directory) as scan:
            for item in scan:
                path = Path(item.path)
                if path.is_file() and path.suffix == ".proto":
                    files.append(path)
                elif path.is_dir():
                    for subitem in detect_files(directory=[item.path, ]):
                        files.append(subitem)
    return files

if __name__ == "__main__":
	# See purchase confirmation emails for details and service_account.json.
    url = "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev"
    api_key = "YOUR_API_KEY"
    encoded_jwt = create_jwt("path/to/service_account.json")
    
    headers = {
        "Authorization": f"Bearer {encoded_jwt}",
        "Content-Type": "application/json" }

    ################################################
    #### Insert, Update, and Remove API Example ####
    ################################################

    # NOTE: See queries.proto for all API definitions.
    # Creating an Insert, Update, or Remove API with JSON. 
    request = {
        # api_key and messages match Insert definition in queries.proto
        "api_key" : api_key,
        "messages": [
            {
                # Part after '/' will always be the proto package and message name
                "@type": "type.googleapis.com/rogue.services.Test",
                
                # Matches the field names for the message
                "attribute1" : 10,
                "attribute2" : 5,
            } ]}

    # No response given. Errors reported in status code.
    # REST call for Insert API.
    response = requests.post(
        f"https://{url}/rest/insert", 
        headers=headers, 
        data=request)
    
    # REST call for Update API.
    response = requests.patch(
        f"https://{url}/rest/update", 
        headers=headers, 
        data=request)
    
    # REST call for Remove API.
    response = requests.delete(
        f"https://{url}/rest/remove", 
        headers=headers, 
        data=request)
    

    ###############################
    ##### Search API Examples #####
    ###############################

    # NOTE: See queries.proto for full API.
    # Example of a basic index query. 
    # For Test, attribute1, attribute2, and attribute3 form the index.
    # Search Query: 
    # Test.attribute1 >= 1 and Test.attribute2 >= 1 and Test.attribute3 >= true
    # AND
    # Test.attribute1 <= 10 and Test.attribute2 <= 10 and Test.attribute3 <= true
    search = {
        "api_key": api_key,
        "queries": [
        {
            "basic": 
            {
                "comparisons": ["GREATER_EQUAL", "LESSER_EQUAL"],
                "operands": [
                {
                    # Part after '/' will always be the proto package and message name
                    "@type": "type.googleapis.com/rogue.services.Test",
                    
                    # Matches the field names for the message
                    "attribute1" : 1,
                    "attribute2" : 1,
                    "attribute3" : True,
                },
                {
                    "@type": "type.googleapis.com/rogue.services.Test",
                    "attribute1" : 10,
                    "attribute2" : 10,
                    "attribute3" : True
                }] 
            }
        }] }

    # All search query types use this URL
    response = requests.get(
        f"https://{url}/rest/search", 
        headers=headers, 
        data=search)
    
    # Queries are zero-indexed. 
    # Results are mapped to that index.
    for result in response.results[0]:
        pass # Will be JSON objects
    
    # Example of a basic non-indexed query.
    # Search Query: Test.attribute1 < 1 Test.attribute2 != 10
    search = {
        "api_key": api_key,
        "queries": [
        {
            "basic": 
            {
                "comparisons": ["LESSER", "NOT_EQUAL"],
                "fields" : [1, 2], # Corresponds to field ids in test.proto
                "operands": [
                {
                    # Part after '/' will always be the proto package and message name
                    "@type": "type.googleapis.com/rogue.services.Test",

                    # Matches the field names for the message
                    "attribute1" : 1,
                },
                {
                    "@type": "type.googleapis.com/rogue.services.Test",
                    "attribute2" : 10,
                }]
            }
        }] }
    
    #####################################
    ##### Schema Change API Example #####
    #####################################
    
    proto_file_definitions = []
    # Proto file definitions. No modifications required.
    for file in detect_files(directories=[
        "absolute/path/to/protos/directory1",
        "absolute/path/to/protos/directory2"]):
        with open(file) as input_file:
            proto_file_definitions.schemas.append(input_file.read())

    # Any schemas excluded will have associated data deleted.
    # Schema change failure results in no changes applied.
    response = requests.post(
        f"https://{url}/rest/search", 
        headers=headers, 
        data={
            "api_key" : api_key,
            "schemas" : proto_file_definitions
        })
#include <format>
#include <fstream>

// External dependencies. Single header files.
#include <cpr/cpr.h>
#include <jwt-cpp/jwt.h>

std::string createJwt()
{
    // Values found in service_account.json.
    const std::string SERVICE_ACCOUNT_EMAIL{ "YOUR_SERVICE_ACCOUNT_EMAIL" };
    const std::string PRIVATE_KEY_ID{ "YOUR_PRIVATE_KEY_ID" };
    const std::string PRIVATE_KEY{ "YOUR_PRIVATE_KEY" };
    const auto now{ std::chrono::system_clock::now() };
    return std::string{ jwt::create()
        .set_issuer(SERVICE_ACCOUNT_EMAIL)
        .set_subject(SERVICE_ACCOUNT_EMAIL)
        .set_audience(std::format(
        	"{}.roguedb.dev", 
            SERVICE_ACCOUNT_EMAIL.substr(
            	0, SERVICE_ACCOUNT_EMAIL.find("@"))))
        .set_issued_at(now)
        .set_expires_at(now + std::chrono::hours(1))
        .set_header_claim("kid", jwt::claim(PRIVATE_KEY_ID))
        .sign(jwt::algorithm::rs256{ PRIVATE_KEY }) };
}

std::vector<std::filesystem::path> detectFiles(
    const std::filesystem::path& directory)
{
    std::vector<std::filesystem::path> files{};
    for(const auto& entry : std::filesystem::directory_iterator{directory})
    {
        if(!std::filesystem::is_directory(entry))
        {
            if(entry.path().extension() == ".proto")
            {
                files.emplace_back(std::move(entry.path()));
            }
        }
        else
        {
            for(auto& filename : detectFiles(entry.path()))
            {
                files.emplace_back(std::move(filename));
            }
        }
    }
    return files;
}

int main(int argc, char** argv)
{
    // DISCLAIMER: We do not currently test C++ with REST.
    // Below code is best effort from documentation of popular
    // single header libraries for REST and JWT.

    // See purchase confirmation emails for details and service_account.json.
	const std::string API_KEY{ "YOUR_API_KEY" };
    const std::string URL{ "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev" };
    const std::string ENCODED_JWT{ createJwt() };
    
    ////////////////////////////////////////////////////////
    ///////  Insert, Update, and Remove API Example  ///////
    ////////////////////////////////////////////////////////

    
    // No response given. Errors reported in status code.
    // REST call for Insert API.

    std::string request{ "{\n\t\"api_key\": \"" };
    request.append(API_KEY);
    request.append(R"(",
  "messages": [
    {
      "@type": "type.googleapis.com/rogue.services.Test",
      "attribute1": 10,
      "attribute2": 5
    }
  ]
})");
    cpr::Response response = cpr::Post(
        cpr::Url{ std::format("{}/rest/insert", URL) },
        cpr::Bearer{ ENCODED_JWT },
        cpr::Header{{ "Content-Type", "application/json" }},
        cpr::Body{ request });

    // REST call for Update API.
    response = cpr::Patch(
        cpr::Url{ std::format("{}/rest/update", URL) },
        cpr::Bearer{ ENCODED_JWT },
        cpr::Header{{ "Content-Type", "application/json" }},
        cpr::Body{ request });

    // REST call for Remove API.
    response = cpr::Delete(
        cpr::Url{ std::format("{}/rest/remove", URL) },
        cpr::Bearer{ ENCODED_JWT },
        cpr::Header{{ "Content-Type", "application/json" }},
        cpr::Body{ request });
    
    //////////////////////////////////////
    ////////  Search API Example  ////////
    //////////////////////////////////////

    // NOTE: See queries.proto for full API.
    // Example of a basic index query. 
    // For Test, attribute1, attribute2, and attribute3 form the index.
    // Search Query: 
    // Test.attribute1 >= 1 and Test.attribute2 >= 1 and Test.attribute3 >= true
    // AND
    // Test.attribute1 <= 10 and Test.attribute2 <= 10 and Test.attribute3 <= true
    request = "{\n\"api_key";
    request.append(API_KEY);

    // For @type, '/' will always be the proto package and message name
    // attribute1, attribute2, and attribute3 match the field names in test.proto
    request.append(R"(",
  "queries": [
    {
      "basic": {
        "comparisons": ["GREATER_EQUAL", "LESSER_EQUAL"],
        "operands": [
          {
            "@type": "type.googleapis.com/rogue.services.Test",
            "attribute1": 1,
            "attribute2": 1,
            "attribute3": true
          },
          {
            "@type": "type.googleapis.com/rogue.services.Test",
            "attribute1": 10,
            "attribute2": 10,
            "attribute3": true
          }
        ]
      }
    }
  ]
})");

    // All search query types use this URL
    // Queries are zero-indexed. 
    // Results are mapped to results field.
    // All messages are stored in the messages field.
    response = cpr::Get(
        cpr::Url{ std::format("{}/rest/search", URL) },
        cpr::Bearer{ ENCODED_JWT },
        cpr::Header{{ "Content-Type", "application/json" }},
        cpr::Body{ request });

    // Example of a basic non-indexed query.
    // Search Query: Test.attribute1 < 1 and Test.attribute2 != 10
    request = "{\n\"api_key";
    request.append(API_KEY);

    // Fields corresponds to the field ids in test.proto
    request.append(R"(",
  "queries": [
    {
      "basic": {
        "comparisons": ["GREATER_EQUAL", "LESSER_EQUAL"],
        "fields": [1, 2],
        "operands": [
          {
            "@type": "type.googleapis.com/rogue.services.Test",
            "attribute1": 1,
          },
          {
            "@type": "type.googleapis.com/rogue.services.Test",
            "attribute1": 10,
          }
        ]
      }
    }
  ]
})");

    ///////////////////////////////////
    //// Schema Change API Example ////
    ///////////////////////////////////

    request = "{\n\"api_key";
    request.append(API_KEY);
    request.append(R"("
    "schemas": [)");

    // All proto files should be sent in a list of
    // their contents. No modifications required.
    for(const auto& file : detectFiles("absolute/path/to/protos/directory"))
    {
        std::ifstream inputFile{ file.native() };
        std::stringstream buffer{};
        buffer << inputFile.rdbuf();
        request.append(std::format("\"{}\",", buffer.str()));
    }
    request.pop_back();
    request.append("]\n}");

    // Any schemas excluded will have associated data deleted.
    // Schema change failure results in no changes applied.
    response = cpr::Post(
        cpr::Url{ std::format("{}/rest/subscribe", URL) },
        cpr::Bearer{ ENCODED_JWT },
        cpr::Header{{ "Content-Type", "application/json" }},
        cpr::Body{ request });
}
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"os"
	"path/filepath"
	"strings"
	"time"

	"github.com/golang-jwt/jwt/v5"
)

func createJWT() string {
	// Values found in service_account.json
	const SERVICE_ACCOUNT_EMAIL = "YOUR_SERVICE_ACCOUNT_EMAIL"
	const PRIVATE_KEY_ID = "YOUR_PRIVATE_KEY_ID"
	const PRIVATE_KEY = "YOUR_PRIVATE_KEY"

	now := time.Now()

	// Create JWT token
	token := jwt.NewWithClaims(jwt.SigningMethodRS256, jwt.MapClaims{
		"iss": SERVICE_ACCOUNT_EMAIL,
		"sub": SERVICE_ACCOUNT_EMAIL,
		"aud": fmt.Sprintf("%s.roguedb.dev", SERVICE_ACCOUNT_EMAIL[:len(SERVICE_ACCOUNT_EMAIL)-len(SERVICE_ACCOUNT_EMAIL[strings.LastIndex(SERVICE_ACCOUNT_EMAIL, "@")+1:])]),
		"iat": now.Unix(),
		"exp": now.Add(time.Hour).Unix(),
		"header": map[string]interface{}{
			"kid": PRIVATE_KEY_ID,
		},
	})

	tokenString, err := token.SignedString([]byte(PRIVATE_KEY))
	if err != nil {
		panic(err)
	}

	return tokenString
}

func detectFiles(directories []string) []string {
	var files []string

	for _, directory := range directories {
		err := filepath.Walk(directory, func(path string, info os.FileInfo, err error) error {
			if err != nil {
				return err
			}

			// Skip directories
			if !info.IsDir() {
				files = append(files, path)
			}

			return nil
		})

		if err != nil {
			panic(err)
		}
	}

	return files
}

func main() {
	// See purchase confirmation emails for details and service_account.json
	const API_KEY = "YOUR_API_KEY"
	const URL = "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev"
	encodedJWT := createJWT()

	////////////////////////////////////////////////////////
	///////  Insert, Update, and Remove API Example  ///////
	////////////////////////////////////////////////////////

	// NOTE: See queries.proto for all API definitions.
	// Creating an Insert, Update, or Remove API with JSON.
	payload := map[string]interface{}{
		"api_key": API_KEY,
		"messages": []map[string]interface{}{
			map[string]interface{}{
				// Part after '/' will always be the proto package and message name
				"@type": "type.googleapis.com/rogue.services.Test",

				// Matches the field names for the message
				"attribute1": 10,
				"attribute2": 5}},
	}

	jsonPayload, err := json.Marshal(payload)
	if err != nil {
		panic(err)
	}

	request, err := http.NewRequest("POST", URL+"/rest/insert", bytes.NewBuffer(jsonPayload)) // Insert API
	// request, err := http.NewRequest("PATCH", URL+"/rest/update", bytes.NewBuffer(jsonPayload)) // Update API
	// request, err := http.NewRequest("DELETE", URL+"/rest/remove", bytes.NewBuffer(jsonPayload)) // Remove API
	if err != nil {
		panic(err)
	}

	request.Header.Set("Authorization", "Bearer "+encodedJWT)
	request.Header.Set("Content-Type", "application/json")

	client := &http.Client{Timeout: 10 * time.Second}

	// No response given. Errors reported in status code.
	response, err := client.Do(request)
	response.Body.Close()

	//////////////////////////////////////
	////////  Search API Example  ////////
	//////////////////////////////////////

	// Example of a baisc index query.
	// For Test, attribute1, attribute2, and attribute3 form the index.
	// Search Query:
	// Test.attribute1 >= 1 and Test.attribute2 >= 1 and Test.attribute3 >= true
	// AND
	// Test.attribute1 <= 10 and Test.attribute2 <= 10 and Test.attribute3 <= true
	payload = map[string]interface{}{
		"api_key": API_KEY,
		"queries": []map[string]interface{}{
			map[string]interface{}{
				"basic": map[string]interface{}{
					"comparisons": []string{"GREATER_EQUAL", "LESSER_EQUAL"},
					"operands": []map[string]interface{}{
						map[string]interface{}{
							// Part after '/' will always be the proto package and message name
							"@type": "type.googleapis.com/rogue.services.Test",

							// Matches the field names for the message
							"attribute1": 1,
							"attribute2": 1,
							"attribute3": true},
						map[string]interface{}{
							"@type":      "type.googleapis.com/rogue.services.Test",
							"attribute1": 10,
							"attribute2": 10,
							"attribute3": true}}}}}}

	jsonPayload, err = json.Marshal(payload)
	if err != nil {
		panic(err)
	}

	// All search query types use this URL
	request, err = http.NewRequest("GET", URL+"/rest/search", bytes.NewBuffer(jsonPayload))
	if err != nil {
		panic(err)
	}

	request.Header.Set("Authorization", "Bearer "+encodedJWT)
	request.Header.Set("Content-Type", "application/json")

	response, err = client.Do(request)
	body, err := io.ReadAll(response.Body)
	if err != nil {
		panic(err)
	}

	// Queries are zero-indexed.
	// Results are mapped to results field.
	// All messages are stored in the messages field.
	fmt.Print(body)
	response.Body.Close()

	// Example of a basic non-indexed query
	// Search Query:
	// Test.attribute1 >= 1 && Test.attribute2 >= 1 && Test.attribute3 >= true
	// AND
	// Test.attribute1 <= 10 && Test.attribute2 <= 10 && Test.attribute3 <= true
	payload = map[string]interface{}{
		"api_key": API_KEY,
		"queries": []map[string]interface{}{
			map[string]interface{}{
				"basic": map[string]interface{}{
					"comparisons": []string{"GREATER_EQUAL", "LESSER_EQUAL"},
					"fields":      []int{1, 2}, // Corresponds to field ids in test.proto
					"operands": []map[string]interface{}{
						map[string]interface{}{
							// Part after '/' will always be the proto package and message name
							"@type": "type.googleapis.com/rogue.services.Test",

							// Matches the field names for the message
							"attribute1": 1},
						map[string]interface{}{
							"@type":      "type.googleapis.com/rogue.services.Test",
							"attribute2": 10}}}}}}

	///////////////////////////////////
	//// Schema Change API Example ////
	///////////////////////////////////

	// All proto files should be sent in a list of
	// their contents. No modifications required.
	var protoFileDefinitions []string
	for _, file := range detectFiles([]string{"path/to/proto/directory1", "path/to/proto/directory2"}) {
		content, err := os.ReadFile(file)
		if err != nil {
			panic(err)
		}
		protoFileDefinitions = append(protoFileDefinitions, string(content))
	}

	// Any schemas excluded will have associated data deleted.
	// Schema change failure results in no changes applied.
	payload = map[string]interface{}{
		"api_key": API_KEY,
		"schemas": protoFileDefinitions}

	jsonPayload, err = json.Marshal(payload)
	if err != nil {
		panic(err)
	}

	request, err = http.NewRequest("POST", URL+"/rest/subscribe", bytes.NewBuffer(jsonPayload))
	if err != nil {
		panic(err)
	}

	request.Header.Set("Authorization", "Bearer "+encodedJWT)
	request.Header.Set("Content-Type", "application/json")

	// No response given. Errors reported in status code.
	response, err = client.Do(request)
	response.Body.Close()
}

Learn how to use gRPC and Protocol Buffers with RogueDB.

Python
C++
Go
import datetime
import json
import jwt
import time
import os
from pathlib import Path

# gRPC and Protocol Buffers Imports
import grpc
from grpc_status import rpc_status

from roguedb.roguedb_pb2_grpc import RogueDBStub
from roguedb.queries_pb2 import (
    Insert, Update, Remove, Basic, Search, 
    LogicalOperator, ComparisonOperator, Subscribe)
from roguedb.test_pb2 import Test

# External Dependency Requirements:
# grpcio, protobuf, PyJWT

def create_jwt(service_account: str, expire_minutes: int = 60) -> str:
    # NOTE: Replace with file path to service_account.json
    # from purchase confirmation email.
    with open(service_account) as input_file:
        key_data = json.load(input_file)

    now = datetime.datetime.now(datetime.timezone.utc)
    iat = int(time.mktime(now.timetuple()))
    exp = int(time.mktime((
        now + datetime.timedelta(minutes=expire_minutes)).timetuple()))

    payload = {
        "iat": iat,
        "exp": exp,
        "iss": key_data['client_email'],
        "aud": f"{key_data['client_email'].split('@')[0]}.roguedb.dev", 
        "sub": key_data['client_email'] }

    headers = {
        "kid": key_data['private_key_id'],
        "alg": "RS256", # Google Cloud service accounts typically use RS256
        "typ": "JWT" }

    return jwt.encode(
        payload, key_data['private_key'], 
        algorithm="RS256", headers=headers)

def detect_files(directories: list[str]) -> list[Path]:
    files = []
    for directory in directories:
        with os.scandir(directory) as scan:
            for item in scan:
                path = Path(item.path)
                if path.is_file() and path.suffix == ".proto":
                    files.append(path)
                elif path.is_dir():
                    for subitem in detect_files(directory=[item.path, ]):
                        files.append(subitem)
    return files

if __name__ == "__main__":
	# See purchase confirmation emails for details and service_account.json.
    url = "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev"
    api_key = "YOUR_API_KEY"
    encoded_jwt = create_jwt("path/to/service_account.json")

    ################################################
    #### Insert, Update, and Remove API Example ####
    ################################################
    
    request = Insert(api_key=api_key) # Insert API
    # update = Update(api_key=api_key) # Update API
    # remove = Remove(api_key=api_key) # Remove API
    
    # Insert, Update, and Remove are identical in use.
    request.messages.add()

    # See test.proto for the schema definition.
    test = Test(attribute1=10)
    request.messages[0].Pack(test)
    
    roguedb = RogueDBStub(
        channel=grpc.secure_channel(f"{url}:443", 
        grpc.ssl_channel_credentials()))

    def yield_request():
        yield request

    try:
        for _ in roguedb.insert(
        # for _ in roguedb.update( # Update API call
        # for _ in roguedb.remove( # Remove API call
            yield_request(), 
            metadata=[("authorization", f"Bearer {encoded_jwt}")]):
            pass
    except grpc.RpcError as rpc_error:
        # Any errors get reported in status.
        status = rpc_status.from_call(rpc_error)
    
    ###############################
    ##### Search API Examples #####
    ###############################

    # Example of a basic index query. 
    # For Test, attribute1, attribute2, and attribute3 form the index.
    # Search Query: 
    # Test.attribute1 >= 1 and Test.attribute2 >= 1 and Test.attribute3 >= true
    # AND
    # Test.attribute1 <= 10 and Test.attribute2 <= 10 and Test.attribute3 <= true
    search = Search(api_key=api_key)
    expression = search.queries.add().basic
    expression.logical_operator = LogicalOperator.AND

    expression.comparisons.append(ComparisonOperator.GREATER_EQUAL)
    expression.operands.add().Pack(Test(attribute1=1, attribute2=1, attribute3=True))

    expression.comparisons.append(ComparisonOperator.LESSER_EQUAL)
    expression.operands.add().Pack(Test(attribute2=10, attribute2=10, attribute3=True))

    def yield_search():
        yield search

    for response in roguedb.search(
        yield_search(),
        metadata=[("authorization", f"Bearer {encoded_jwt}")]):
        results = []

        # Queries are zero-indexed. Partial results get sent
        # and mapped to that index.
        for result in response.results[0].messages:
            test = Test()
            response.results[0].Unpack(test)
        
        # Each response sends a list of the query ids finished
        # with processing.
        if 0 in response.finished:
            pass # Finished processing.

    # Example of a basic non-indexed query. 
    # Search Query: Test.attribute1 < 1 AND Test.attribute2 != 10
    search = Search(api_key=api_key)
    expression = search.queries.add().basic
    expression.logical_operator = LogicalOperator.AND

    expression.comparisons.append(ComparisonOperator.LESSER)
    expression.operands.add().Pack(Test(attribute1=1))
    expression.fields.append(1) # Corresponds to field id in test.proto

    expression.comparisons.append(ComparisonOperator.NOT_EQUAL)
    expression.operands.add().Pack(Test(attribute2=10))
    expression.fields.append(2) # Corresponds to field id in test.proto

    #####################################
    ##### Schema Change API Example #####
    #####################################

    subscribe = Subscribe(api_key=api_key)

    # All proto files should be sent in a list of
    # their contents. No modifications required.
    for file in detect_files(directories=[
        "absolute/path/to/protos/directory1",
        "absolute/path/to/protos/directory2"]):
        with open(file) as input_file:
            subscribe.schemas.append(input_file.read())

    # Any schemas excluded will have associated data deleted.
    # Schema change failure results in no changes applied.
    response = roguedb.subscribe(subscribe)
#include <format>
#include <fstream>
#include <grpcpp/grpcpp.h>
#include <jwt-cpp/jwt.h>

#include "getting_started/roguedb.grpc.pb.h"

std::string createJwt()
{
    // Values found in service_account.json.
    const std::string SERVICE_ACCOUNT_EMAIL{ "YOUR_SERVICE_ACCOUNT_EMAIL" };
    const std::string PRIVATE_KEY_ID{ "YOUR_PRIVATE_KEY_ID" };
    const std::string PRIVATE_KEY{ "YOUR_PRIVATE_KEY" };
    const auto now{ std::chrono::system_clock::now() };
    return std::string{ jwt::create()
        .set_issuer(SERVICE_ACCOUNT_EMAIL)
        .set_subject(SERVICE_ACCOUNT_EMAIL)
        .set_audience(std::format(
        	"{}.roguedb.dev", 
            SERVICE_ACCOUNT_EMAIL.substr(
            	0, SERVICE_ACCOUNT_EMAIL.find("@"))))
        .set_issued_at(now)
        .set_expires_at(now + std::chrono::hours(1))
        .set_header_claim("kid", jwt::claim(PRIVATE_KEY_ID))
        .sign(jwt::algorithm::rs256{ PRIVATE_KEY }) };
}

std::vector<std::filesystem::path> detectFiles(
    const std::filesystem::path& directory)
{
    std::vector<std::filesystem::path> files{};
    for(const auto& entry : std::filesystem::directory_iterator{directory})
    {
        if(!std::filesystem::is_directory(entry))
        {
            if(entry.path().extension() == ".proto")
            {
                files.emplace_back(std::move(entry.path()));
            }
        }
        else
        {
            for(auto& filename : detectFiles(entry.path()))
            {
                files.emplace_back(std::move(filename));
            }
        }
    }
    return files;
}

int main(int argc, char** argv)
{
    // See purchase confirmation emails for details and service_account.json.
	const std::string API_KEY{ "YOUR_API_KEY" };
    const std::string URL{ "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev" };
    const std::string ENCODED_JWT{ createJwt() };
    
    ////////////////////////////////////////////////////////
    ///////  Insert, Update, and Remove API Example  ///////
    ////////////////////////////////////////////////////////
    
    rogue::services::Test test{};
    test.set_attribute1(10);
    
    rogue::services::Insert request{}; // Insert API
    // rogue::services::Update update{}; // Update API
    // rogue::services::Remove remove{}; // Remove API
    
    request.set_api_key(API_KEY);

    // Insert, Update, and Remove are identical in use.
    request.add_messages()->PackFrom(test);
    
    // gRPC Connection
    std::unique_ptr<rogue::services::RogueDB::Stub> roguedb{ rogue::services::RogueDB::NewStub(
    grpc::CreateChannel(
        std::format("{}:443", URL),
        grpc::SslCredentials(grpc::SslCredentialsOptions()))) };
        
    grpc::ClientContext context{};
    context.AddMetadata("Authorization", std::format("Bearer {}", ENCODED_JWT));
    auto stream{ roguedb->insert(&context) };
    // auto stream{ roguedb->update(&context) }; // Update API
    // auto stream{ roguedb->remove(&context) }; // Remove API

    stream->Write(request);
    stream->WritesDone(); // Signal all queries sent. Otherwise, blocks.
    
    // No response is given for Insert, Update, and Remove
    // Any errors get reported in status.
    grpc::Status status{ stream->Finish() };
    
    //////////////////////////////////////
    ////////  Search API Example  ////////
    //////////////////////////////////////

    {
        // Example of a baisc index query.
        rogue::services::Search search{};
        search.set_api_key(API_KEY);
        
        // For Test, attribute1, attribute2, and attribute3 form the index.
        // Search Query:
        // Test.attribute1 >= 1 and Test.attribute2 >= 1 and Test.attribute3 >= true
        // AND
        // Test.attribute1 <= 10 and Test.attribute2 <= 10 and Test.attribute3 <= true
        rogue::services::Basic& expression{ *search.add_queries()->mutable_basic() };
        expression.set_logical_operator(rogue::services::LogicalOperator::AND);

        test = rogue::services::Test{};
        expression.add_comparisons(rogue::services::ComparisonOperator::GREATER_EQUAL);
        test.set_attribute1(1);
        test.set_attribute2(1);
        test.set_attribute3(true);
        (*expression.add_operands()).PackFrom(test);
            
        expression.add_comparisons(rogue::services::ComparisonOperator::LESSER_EQUAL);
        test.set_attribute1(10);
        test.set_attribute2(10);
        test.set_attribute3(true);
        (*expression.add_operands()).PackFrom(test);

        grpc::ClientContext context{};
        rogue::services::Response response{};
        std::shared_ptr<grpc::ClientReaderWriter<
            rogue::services::Search, rogue::services::Response>> stream{
                roguedb->search(&context) };
            
        stream->Write(search);
        stream->WritesDone(); // Signal all queries sent. Otherwise, blocks.

        // Multiple queries or large queries should be processed on a separate thread
        // to prevent blocking the database.
        stream->Read(&response);
        status = stream->Finish(); // Signal all queries sent. Blocks otherwise.

        std::vector<rogue::services::Test> results{};
        // Queries are zero-indexed. Partial results get sent
        // and mapped to that index.
        for(const auto& result : response.results().at(0).messages())
        {
            result.UnpackTo(&results.emplace_back());
        }

        // Each response sends a list of the query ids finished
        // with processing.
        for(const auto& finishedId : response.finished())
        {
            if(finishedId == 0)
            {}
        }
    }
    {
        // Example of a baisc non-indexed query.
        rogue::services::Search search{};
        search.set_api_key(API_KEY);
        
        // Search Query: 
        // Test.attribute1 >= 1 && Test.attribute2 <= 10
        rogue::services::Basic& expression{ *search.add_queries()->mutable_basic() };
        expression.set_logical_operator(rogue::services::LogicalOperator::AND);

        test = rogue::services::Test{};
        test.set_attribute1(1);
        expression.add_fields(1); // Corresponds to field id in test.proto
        (*expression.add_operands()).PackFrom(test);
        expression.add_comparisons(rogue::services::ComparisonOperator::GREATER_EQUAL);
            
        test.set_attribute2(10);
        expression.add_fields(2); // Corresponds to field id in test.proto
        (*expression.add_operands()).PackFrom(test);
        expression.add_comparisons(rogue::services::ComparisonOperator::LESSER_EQUAL);
    }

    ///////////////////////////////////
    //// Schema Change API Example ////
    ///////////////////////////////////

    rogue::services::Subscribe subscribe{};
    subscribe.set_api_key(API_KEY);

    // All proto files should be sent in a list of
    // their contents. No modifications required.
    for(const auto& file : detectFiles("absolute/path/to/protos/directory"))
    {
        std::ifstream inputFile{ file.native() };
        std::stringstream buffer{};
        buffer << inputFile.rdbuf();
        subscribe.add_schemas(buffer.str());
    }

    rogue::services::Response response{};
    // Any schemas excluded will have associated data deleted.
    // Schema change failure results in no changes applied.
    roguedb->subscribe(&context, subscribe, &response);
}
package main

import (
	"context"
	"fmt"
	"io"
	"os"
	"path/filepath"
	"slices"
	"strings"
	"time"

	"github.com/golang-jwt/jwt/v5"
	"github.com/roguedb/roguedb-open-source/roguedb/core_schemas"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
	"google.golang.org/protobuf/types/known/anypb"
)

func createJWT() string {
	// Values found in service_account.json
	const SERVICE_ACCOUNT_EMAIL = "YOUR_SERVICE_ACCOUNT_EMAIL"
	const PRIVATE_KEY_ID = "YOUR_PRIVATE_KEY_ID"
	const PRIVATE_KEY = "YOUR_PRIVATE_KEY"

	now := time.Now()

	// Create JWT token
	token := jwt.NewWithClaims(jwt.SigningMethodRS256, jwt.MapClaims{
		"iss": SERVICE_ACCOUNT_EMAIL,
		"sub": SERVICE_ACCOUNT_EMAIL,
		"aud": fmt.Sprintf("%s.roguedb.dev", SERVICE_ACCOUNT_EMAIL[:len(SERVICE_ACCOUNT_EMAIL)-len(SERVICE_ACCOUNT_EMAIL[strings.LastIndex(SERVICE_ACCOUNT_EMAIL, "@")+1:])]),
		"iat": now.Unix(),
		"exp": now.Add(time.Hour).Unix(),
		"header": map[string]interface{}{
			"kid": PRIVATE_KEY_ID,
		},
	})

	tokenString, err := token.SignedString([]byte(PRIVATE_KEY))
	if err != nil {
		panic(err)
	}

	return tokenString
}

func detectFiles(directories []string) []string {
	var files []string

	for _, directory := range directories {
		err := filepath.Walk(directory, func(path string, info os.FileInfo, err error) error {
			if err != nil {
				return err
			}

			// Skip directories
			if !info.IsDir() {
				files = append(files, path)
			}

			return nil
		})

		if err != nil {
			panic(err)
		}
	}

	return files
}

func main() {
	// See purchase confirmation emails for details and service_account.json
	const API_KEY = "YOUR_API_KEY"
	const URL = "c-[YOUR_IDENTIFIER_FIRST_28_CHARACTERS].roguedb.dev"
	encodedJWT := createJWT()

	////////////////////////////////////////////////////////
	///////  Insert, Update, and Remove API Example  ///////
	////////////////////////////////////////////////////////

	connection, err := grpc.NewClient(URL, grpc.WithTransportCredentials(credentials.NewTLS(nil)))
	defer connection.Close()
	client := core_schemas.NewRogueDBClient(connection)

	md := metadata.Pairs("authorization", fmt.Sprintf("Bearer: %s", encodedJWT))
	ctx := metadata.NewOutgoingContext(context.Background(), md)

	stream, err := client.Insert(ctx) // Insert API
	// stream, err = client.Update(ctx) // Update API
	// stream, err = client.Remove(ctx) // Remove API
	if err != nil {
		panic(err)
	}

	complete := make(chan struct{})

	// No response is given for Insert, Update, and Remove
	// Any errors get reported in status.
	go func() {
		for {
			_, err := stream.Recv()
			if err == io.EOF {
				close(complete)
				return
			}

			if err != nil {
				panic(err)
			}
		}
	}()

	request := core_schemas.Insert{ApiKey: API_KEY}
	// request := core_schemas.Update{ApiKey: API_KEY} // Update API
	// request := core_schemas.Remove{ApiKey: API_KEY} // Remove API

	// Insert, Update, and Remove are identical in use.
	request.Messages = append(request.Messages, &anypb.Any{})
	test := core_schemas.Test{Attribute1: 10}
	err = request.Messages[0].MarshalFrom(&test)
	if err != nil {
		panic(err)
	}

	err = stream.Send(&request)
	if err != nil {
		panic(err)
	}

	err = stream.CloseSend() // Signal all queries sent. Otherwise, blocks.
	if err != nil {
		panic(err)
	}

	<-complete

	//////////////////////////////////////
	////////  Search API Example  ////////
	//////////////////////////////////////

	// Example of a baisc index query.
	searchStream, err := client.Search(ctx)
	if err != nil {
		panic(err)
	}

	// Search Query:
	// Test.attribute1 >= 1 && Test.attribute2 >= 1 && Test.attribute3 >= true
	// AND
	// Test.attribute1 <= 10 && Test.attribute2 <= 10 && Test.attribute3 <= true
	search := core_schemas.Search{ApiKey: API_KEY}
	search.Queries = append(search.Queries, &core_schemas.Expression{})
	basic := search.Queries[0].GetBasic()
	basic.LogicalOperator = core_schemas.LogicalOperator_AND

	basic.Comparisons = append(basic.Comparisons, core_schemas.ComparisonOperator_GREATER_EQUAL)
	basic.Operands = append(basic.Operands, &anypb.Any{})
	test = core_schemas.Test{Attribute1: 1, Attribute2: 1, Attribute3: true}
	basic.Operands[0].MarshalFrom(&test)

	basic.Comparisons = append(basic.Comparisons, core_schemas.ComparisonOperator_LESSER_EQUAL)
	basic.Operands = append(basic.Operands, &anypb.Any{})
	test = core_schemas.Test{Attribute1: 10, Attribute2: 10, Attribute3: true}
	basic.Operands[1].MarshalFrom(&test)

	complete = make(chan struct{})

	// Multiple queries or large queries should be processed on a separate thread
	// to prevent blocking the database.
	go func() {
		for {
			response, err := stream.Recv()
			if err == io.EOF {
				close(complete)
				return
			}

			if err != nil {
				panic(err)
			}

			// Queries are zero-indexed. Partial results get sent
			// and mapped to that index.
			for _, message := range response.Results[0].Messages {
				temp := core_schemas.Test{}
				message.UnmarshalTo(&temp)
			}

			// Each response sends a list of the query ids finished
			// with processing.
			if slices.Contains(response.Finished, 0) {
			}
		}
	}()

	err = searchStream.Send(&search)
	if err != nil {
		panic(err)
	}
	err = searchStream.CloseSend() // Signal all queries sent. Otherwise, blocks.
	if err != nil {
		panic(err)
	}
	<-complete

	// Example of a baisc non-indexed query.
	search = core_schemas.Search{ApiKey: API_KEY}
	search.Queries = nil

	// Search Query:
	// Test.attribute1 >= 1 && Test.attribute2 <= 10
	search.Queries = append(search.Queries, &core_schemas.Expression{})
	basic = search.Queries[0].GetBasic()
	basic.LogicalOperator = core_schemas.LogicalOperator_AND
	basic.Fields = append(basic.Fields, 1, 2) // Corresponds to field ids in test.proto

	basic.Comparisons = append(basic.Comparisons, core_schemas.ComparisonOperator_GREATER_EQUAL)
	basic.Operands = append(basic.Operands, &anypb.Any{})
	test = core_schemas.Test{Attribute1: 1}
	basic.Operands[0].MarshalFrom(&test)

	basic.Comparisons = append(basic.Comparisons, core_schemas.ComparisonOperator_LESSER_EQUAL)
	basic.Operands = append(basic.Operands, &anypb.Any{})
	test = core_schemas.Test{Attribute2: 10}
	basic.Operands[1].MarshalFrom(&test)

	///////////////////////////////////
	//// Schema Change API Example ////
	///////////////////////////////////

	subscribe := core_schemas.Subscribe{ApiKey: API_KEY}

	// All proto files should be sent in a list of
	// their contents. No modifications required.
	for _, file := range detectFiles([]string{"path/to/proto/directory1", "path/to/proto/directory2"}) {
		content, err := os.ReadFile(file)
		if err != nil {
			panic(err)
		}
		subscribe.Schemas = append(subscribe.Schemas, string(content))
	}

	// Any schemas excluded will have associated data deleted.
	// Schema change failure results in no changes applied.
	_, err = client.Subscribe(ctx, &subscribe)
	if err != nil {
		panic(err)
	}
}

Proto service and API definitions for RogueDB.

Proto
syntax = "proto3";

import "google/api/annotations.proto";
import "google/protobuf/any.proto";

package rogue.services;
option go_package = "roguedb/core_schemas";

service RogueDB
{
    // gRPC APIs
	rpc insert(stream Insert) returns(stream Response) {}
    rpc update(stream Update) returns(stream Response) {}
    rpc remove(stream Remove) returns(stream Response) {}
	rpc search(stream Search) returns(stream Response) {}
    rpc subscribe(Subscribe) returns(Response) {
        option (google.api.http) = {
            post: "/rest/subscribe",
            body: "*" }; }

    // REST APIs
    rpc rest_insert(Insert) returns(Response) {
        option (google.api.http) = {
            post: "/rest/insert",
            body: "*" }; }

    rpc rest_update(Update) returns(Response) {
        option (google.api.http) = {
            patch: "/rest/update",
            body: "*" }; }

    rpc rest_remove(Remove) returns(Response) {
        option (google.api.http) = {
            delete: "/rest/remove",
            body: "*" }; }

    rpc rest_search(Search) returns(stream Response) {
        option (google.api.http) = {
            get: "/rest/search",
            body: "*" }; }

    // NOTE: Only used for benchmarking.
    rpc complete(Insert) returns(Response) {}
}

message Update
{
    string api_key = 1;
    repeated google.protobuf.Any messages = 2;
}

message Remove
{
    string api_key = 1;
    repeated google.protobuf.Any messages = 2;
}

message Insert
{
    string api_key = 1;
    repeated google.protobuf.Any messages = 2;
}

message Search
{
    string api_key = 1;
    repeated Expression queries = 2;
}

message Subscribe
{
    string api_key = 1;
    repeated string schemas = 2;
}

message Response
{
    map<uint64, Messages> results = 1;
    repeated uint64 finished = 2;
}

message Messages
{
    repeated google.protobuf.Any messages = 1;
}

enum LogicalOperator
{
    AND = 0;
    OR = 1;
}
enum ComparisonOperator
{
    COMPARISON_OPERATOR_UNKNOWN = 0;
    EQUAL = 1;
    NOT_EQUAL = 2;
    LESSER = 3;
    LESSER_EQUAL = 4;
    GREATER = 5;
    GREATER_EQUAL = 6;
}

message Basic
{
    LogicalOperator logical_operator = 1;
    repeated ComparisonOperator comparisons = 2;
    repeated google.protobuf.Any operands = 3;
    repeated int32 fields = 4; // Future feature.
}

// The below will be available in the upcoming release.
// Tentative design outlined below.
// Subject to change without notice.
message Complex
{
    LogicalOperator logical_operator = 1;
    repeated Expression expressions = 2;
}

message Expression
{
    oneof expression
    {
        Basic basic = 1;
        Complex complex = 2;
        Compositional compositional = 3;
    }
}

message Mapping
{
    string identifier = 1;
    repeated uint32 input_fields = 2;
    repeated uint32 output_fields = 3;
    Expression expression = 4;
}

message Compositional
{
    repeated Mapping mappings = 1;
}

enum Permission
{
    NO_PERMISSIONS = 0;
    EXECUTE = 1;
    WRITE = 2;
    WRITE_EXECUTE = 3;
    READ = 4;
    READ_EXECUTE = 5;
    READ_WRITE = 6;
    READ_WRITE_EXECUTE = 7;
}

message User
{
    string api_key = 1; // index-1
    map<string, Permission> permissions = 2;
    bool admin = 3;
}

// Use this for testing a connection.
message Test
{
    int32 attribute1 = 1; // index-1
    int64 attribute2 = 2; // index-2
    bool attribute3 = 3; // index-3
    string attribute4 = 4;
    uint32 attribute5 = 5;
    uint64 attribute6 = 6;
    double attribute7 = 7;
    float attribute8 = 8;
    sint32 attribute9 = 9;
    sint64 attribute10 = 10;
    fixed32 attribute11 = 11;
    fixed64 attribute12 = 12;
    sfixed32 attribute13 = 13;
    sfixed64 attribute14 = 14;
    bytes attribute15 = 15;
    repeated int32 attribute16 = 16;
    map<uint32, string> attribute17 = 17;
}

‍

Schema Management

RogueDB uses Protocol Buffers for schema definitions.

‍

Protobuf
syntax = "proto3";

import "google/protobuf/timestamp.proto";

// Every message must be indexed to use with the CRUD APIs.
// This includes nested messages, excluding Google messages.
// Test, User, and Unregistered are reserved for internal use.
// Message names must be unique, but are case sensitive.
message Aggregate
{
    string symbol = 1; // index-1 This part is not included for index order.
    google.protobuf.Timestamp timestamp = 2; // index-3z Non-numerical characters allowed.
    uint32 aggregate_period = 3; //index-2 No space between "//" and "index" is also valid.
    float open = 4;
    float close = 5;
    float low = 6;
    float high = 7;
    uint32 volume = 8;
}

// NOTE: The order of evaluation for the index is symbol -> aggregate_period -> timestamp. 

See Index Performance below for general guidelines on the best way to create an index.

‍

Special Note: Using Protocol Buffer's compiler protoc to generate messages in your target language allows validation locally before applying changes to the database.

‍

Authentication and Identification

The complete example includes everything needed for secure connection with HTTPS using a JWT token for authentication and an API key for identification.

‍

No additional setup is required.

‍

Secure connections are the only allowed connections. Any connection not containing all three components are rejected by default.

‍

User Roles and Permissions

The API key from the purchase confirmation email has all permissions by default and cannot be revoked.

‍

RogueDB supports the creation of users and assigning of permissions per schema. Users have no permissions by default. The following permissions can be assigned:

Read: Permission for the Search API.
Write: Permission for the Create, Update, and Remove APIs.
Execute: Permission for the Subscribe API.
Admin: Grants all permissions and overrides any existing permission.

Manage users through the CRUD APIs:

‍

Python
C++
from roguedb.queries_pb2 import Insert
from roguedb.user_pb2 import User, Permission

# See queries.proto and user.proto for the full API.
insert = Insert(api_key=api_key)
user = User(api_key="CUSTOMER_MANAGED_KEY", admin=False)

# The key is the schema and case sensitive.
# Assign any combination of the three permissions.
user.permissions["Test"] = Permission.READ_WRITE_EXECUTE
insert.messages.add().Pack(user)

def yield_insert():
	yield insert

response = roguedb.insert(
    yield_insert(), 
    metadata=[("authorization", f"Bearer {encoded_jwt}")])

response = requests.post(
    f"https://{url}/rest/insert", 
    headers=headers, 
    data={
        "api_key" : api_key,
        "messages" : [
            {
                "@type": "type.googleapis.com/rogue.services.User",
                "api_key" : "USER_MANAGED_API_KEY",
                "permissions": { "Test": "READ_WRITE_EXECUTE" }
            }
        ]
    })
// #include "roguedb/queries.pb.h"
// #include "roguedb/user.pb.h"

Insert insert{};
*insert.mutable_credentials() = credentials;

Rogue::Services::User user{};
user.set_api_key("CUSTOMER_MANAGED_KEY");
user.set_admin(false);
(*user.mutable_permissions())["user"] = Rogue::Services::Permission::READ;
    
*insert.add_messages() = std::move(user);

grpc::Context context{};
std::unique_ptr<grpc::ClientReaderWriter<rogue::services::Insert, rogue::services::Response>> stream{ 
    writeStub->insert(&context) };
stream->Write(query);

Index Performance

Partitioning: Example - Use stock symbol over timestamp as the first index.
- RogueDB optimizes insertion for "bucketed" data versus incremental based index strategies.
Granularity: Example - Use stock symbol over aggregate period as the first index.
- RogueDB optimizes for multi-field indexing with negligible effects on performance.
Minimal: Example - Use the smallest reasonable number of fields to form a unique index.
- RogueDB optimizes memory footprint with internal metadata utilizing the subset of fields identified for indexing.
Order: Arrange categorical fields first and incremental fields last. Only include fields used in all indexed search queries.

Practical use cases means swapping the order typically used in other databases for the fields being indexed. Categorization fields go first. Incrementing fields go last. By doing this, RogueDB can leverage the natural bucketing of data to parallelize operations.

‍

Permission Propogation

Permissions flow downwards from the parent schema. For instance, consider the following example:

‍

1syntax = "proto3";
2
3message MedicalRecords
4{
5    // Stores HIPPA related items of a customer.
6}
7
8message FinancialRecords
9{
10    // Stores financial related items of a customer.
11}
12
13message Customer
14{
15    MedicalRecords medical = 1;
16    FinancialRecords financial = 2;
17    // Stores additional personally identifiable information (PII).
18}

Now presume we have a user, Foo, with the following permissions:

MedicalRecords: No permissions by default.
FinancialRecords: Read permissions.
Customer: Write and Read permissions.

Due to the composition of Customer including MedicalRecords and FinancialRecords, Foo can do the following when using Customer as the record:

MedicalRecords: No permissions by default => Write and Read permissions.
FinancialRecords: Read permissions => Write and Read permissions.

Careful consideration needs to be made when designing schemas to ensure permissions do not propagate in unexpected ways. As such, the Execute permission and Admin role should be given sparingly, if at all, and under additional security mechanisms.
‍

Design Suggestion

Creation of a unique ID per customer with storage of MedicalRecords, FinancialRecords, and PII as separate schemas in the database prevents propagation of permissions. This further streamlines efficient storage and retrieval of data to only the required components versus the entirety of a record. UUIDs are an excellent mechanism for unique identifiers.

‍

Consider the following as a more secure alternative:

1syntax = "proto3";
2
3message MedicalRecords
4{
5	string customer_id = 1;
6    // Stores HIPPA related items of a customer.
7}
8
9message FinancialRecords
10{
11	string customer_id = 1;
12    // Stores financial related items of a customer.
13}
14
15message PII
16{
17    string customer_id = 1;
18    // Stores additional personally identifiable information (PII).
19}

Special Considerations

Duplicate Data: RogueDB automatically removes all duplicate data based on the index.
Highly Nested Messages: Consider separating into smaller individual components.
- RogueDB does not intend to support queries that allow specification into nested message fields. If the field's indexing strategy does not cover intended use case, the message will need to be separated.
- Protobuf serialization becomes slower for deeply nested messages due to the recursive nature. Simpler messages means faster performance.
CRUD Operations: Round robin scheduling of CRUD operations.
- RogueDB utilitizes batching of operations to increase throughput.
- Order of execution: Create, Update, Remove, Basic Queries, Complex Queries, Compositional Queries
Order Guarantees:
- Ordering is not guaranteed between CRUD operations or requests.
- Duplicate messages based on index in a request only uses the first message.