Skip to main content

Trial Datastore

caution

This module is still in active development and should be considered a prerelease version.

Cogment Trial Datastore implements the datalog API. It is designed to store the trajectory samples generated by the trials for online and offline usage.

The Trial Datastore can work in two modes:

  • Memory storage, which is the default mode, stores the data in memory up to a maximum size and then evicts the least recently used samples.
  • File storage stores the data on disk in a file. The storage engine relies on bbolt.

Command line

The Trial Datastore is simply called this way

$ cogment services trial_datastore --port=9000 --file_storage=./database.db

Configuration

The Trial Datastore configuration can be specified either through the command line or environment variables.

port

The TCP port where to serve the The Data/Log API and the Trial Datastore API. The former is where the orchestrator connects to output the data generated by trials, the latter is the API that can be used to retrieve online or offline stored data.

Can be specified as:

  • a command line option, e.g. --port=12000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_PORT=12000,
  • its default value is 9003.

grpc_reflection

Whether or not to enable gRPC reflection on the served endpoints.

Can be specified as:

  • a command line flag, e.g. --grpc_reflection,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_GRPC_REFLECTION=1,
  • by default, it is disabled.

memory_storage_max_samples_size

This defines the maximum cumulated size of samples the memory storage holds before evicting least recently used trials samples. It is defined in bytes.

Can be specified as:

  • a command line option, e.g. --memory_storage_max_samples_size=100000000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_MEMORY_STORAGE_MAX_SAMPLE_SIZE=100000000,
  • its default value is 1073741824 Bytes, i.e. 1GB.

file_storage

Path to the database file used by the file storage. If provided, the trial datastore will use the file storage.

Can be specified as:

  • a command line flag, e.g. --file_storage, to use the default value .cogment/trial_datastore.db,
  • a command line option, e.g. --file_storage=./path/to/trial_datastore.db,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_FILE_STORAGE_PATH=./path/to/trial_datastore.db,
  • it has no default value.

log_level

Set to define the minimum level for logging. Possible values are: off, error, warning, info, debug, trace. Note however that all trace and most debug level logs will only output if running the debug compiled version of the Orchestrator.

Can be specified as:

  • a command line option, e.g. --log_level=debug,
  • an environment variable, e.g. COGMENT_LOG_LEVEL=5,
  • default value is info.

log_file

Base file name for daily log output. The name will be suffixed with the date and a new file will be made every day. If not provided the logs go to stdout.

Can be specified as:

  • a command line option, e.g. --log_file=./path/to/cogment.log,
  • an environment variable, e.g. COGMENT_LOG_FILE=./path/to/cogment.log,
  • default value is info.