Skip to main content

Trial Datastore Server

caution

This module is still in active development and should be considered a prerelease version.

Cogment Trial Datastore implements the datalog API. It is designed to store the trajectory samples generated by the trials for online and offline usage.

The Cogment CLI also includes an implementation of a client.

The Trial Datastore service included in the Cogment CLI can work in two modes:

  • Memory storage, which is the default mode, stores the data in memory up to a maximum size and then evicts the least recently used samples.
  • File storage stores the data on disk in a file. The storage engine relies on bbolt.

The Trial Datastore service is simply called this way

$ cogment services trial_datastore --port=9000 --file_storage=./database.db

Configuration

The Trial Datastore configuration can be specified either through the command line or environment variables.

port

The TCP port where to offer gRPC services. The Trial Datastore offers two services at the same TCP port: the Datalog API and the Trial Datastore API. The Datalog API is where the Datastore receives the raw trial data from the Orchestrator. The Trial Datastore API is is where the Datastore delivers online or offline stored data (typically to actors for training).

Can be specified as:

  • a command line option, e.g. --port=12000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_PORT=12000,
  • its default value is 9003.

grpc_reflection

Whether or not to enable gRPC reflection on the served endpoints.

Can be specified as:

  • a command line flag, e.g. --grpc_reflection,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_GRPC_REFLECTION=1,
  • by default, it is disabled.

memory_storage_max_samples_size

This defines the maximum cumulated size of samples the memory storage holds before evicting least recently used trials samples. It is defined in bytes.

Can be specified as:

  • a command line option, e.g. --memory_storage_max_samples_size=100000000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_MEMORY_STORAGE_MAX_SAMPLE_SIZE=100000000,
  • its default value is 1073741824 Bytes, i.e. 1GB.

file_storage

Path to the database file used by the file storage. If provided, the trial datastore will use the file storage.

Can be specified as:

  • a command line flag, e.g. --file_storage, to use the default value .cogment/trial_datastore.db,
  • a command line option, e.g. --file_storage=./path/to/trial_datastore.db,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_FILE_STORAGE_PATH=./path/to/trial_datastore.db,
  • it has no default value.

log_level

Set to define the minimum level for logging. Possible values are: off, error, warning, info, debug, trace. Note however that all trace and most debug level logs will only output if running the debug compiled version of the Orchestrator.

Can be specified as:

  • a command line option, e.g. --log_level=debug,
  • an environment variable, e.g. COGMENT_LOG_LEVEL=5,
  • default value is info.

log_file

Base file name for daily log output. The name will be suffixed with the date and a new file will be made every day. If not provided the logs go to stdout.

Can be specified as:

  • a command line option, e.g. --log_file=./path/to/cogment.log,
  • an environment variable, e.g. COGMENT_LOG_FILE=./path/to/cogment.log,
  • default value is info.