Skip to main content

Trial Datastore Server

Cogment Trial Datastore implements the datalog API. It is designed to store the trajectory samples generated by the trials for online and offline usage.

The Cogment CLI also includes an implementation of a client.

The Trial Datastore service included in the Cogment CLI can work in two modes:

  • Memory storage, which is the default mode, stores the data in memory up to a maximum size and then evicts the least recently used samples.
  • File storage stores the data on disk in a file. The storage engine relies on bbolt.

The Trial Datastore service is simply called this way

$ cogment services trial_datastore --port=9000 --file_storage=./database.db

Configuration​

The Trial Datastore configuration can be specified either through the command line or environment variables.

port​

The TCP port where to offer gRPC services. The Trial Datastore offers two services at the same TCP port: the Datalog API and the Trial Datastore API. The Datalog API is where the Datastore receives the raw trial data from the Orchestrator. The Trial Datastore API is is where the Datastore delivers online or offline stored data (typically to actors for training).

If set to 0, then the system will automatically choose a free port. This is normally used in combination with a Directory.

Can be specified as:

  • a command line option, e.g. --port=12000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_PORT=12000,
  • its default value is 9003.

grpc_reflection​

Whether or not to enable gRPC reflection on the served endpoints.

Can be specified as:

  • a command line flag, e.g. --grpc_reflection,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_GRPC_REFLECTION=1,
  • by default, it is disabled.

memory_storage_max_samples_size​

This defines the maximum cumulated size of samples the memory storage holds before evicting least recently used trials samples. It is defined in bytes.

Can be specified as:

  • a command line option, e.g. --memory_storage_max_samples_size=100000000,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_MEMORY_STORAGE_MAX_SAMPLE_SIZE=100000000,
  • its default value is 1073741824 Bytes, i.e. 1GB.

file_storage​

Path to the database file used by the file storage. If provided, the trial datastore will use the file storage.

Can be specified as:

  • a command line flag, e.g. --file_storage, to use the default value .cogment/trial_datastore.db,
  • a command line option, e.g. --file_storage=./path/to/trial_datastore.db,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_FILE_STORAGE_PATH=./path/to/trial_datastore.db,
  • it has no default value.

log_level​

Set to define the minimum level for logging. Possible values are: off, error, warning, info, debug, trace. Note however that all trace and most debug level logs will only output if running the debug compiled version of the Orchestrator.

Can be specified as:

  • a command line option, e.g. --log_level=debug,
  • an environment variable, e.g. COGMENT_LOG_LEVEL=5,
  • default value is info.

log_file​

Base file name for daily log output. The name will be suffixed with the date and a new file will be made every day. If not provided the logs go to stdout.

Can be specified as:

  • a command line option, e.g. --log_file=./path/to/cogment.log,
  • an environment variable, e.g. COGMENT_LOG_FILE=./path/to/cogment.log,
  • default value is info.

directory_endpoint​

Cogment endpoint of the directory service. It must be a gRPC endpoint. The directory will be used to register the trial datastore services for discovery by other services. If not provided, the trial datastore will not auto register, in which case manual registration to the directory must be done, or an explicit address must be provided to access the services of the trial datastore.

Can be specified as:

  • a command line option, e.g. --directory_endpoint=grpc://foo:9005,
  • an environment variable, e.g. COGMENT_DIRECTORY_ENDPOINT=grpc://foo:9005,
  • it has no default value.

directory_authentication_token​

Authentication token for services registered in the Directory. It is recorded in the Directory when registering a service. And a matching token must be provided to inquire for the service. An empty token is the same as no token.

Can be specified as:

  • a command line option, e.g. --directory_authentication_token=GH670ploT,
  • an environment variable, e.g. COGMENT_DIRECTORY_AUTHENTICATION_TOKEN=GH670ploT,
  • it has no default value.

directory_registration_host​

This is the host that will be registered to the Directory for the Trial Datastore services. If not provided, the Trial Datastore will determine its own IP address and use that as the registration host.

In some circumstances, the IP address determined by Cogment may be wrong (e.g. multiple interfaces, load balancing, firewall), thus a host (hostname or IP address) must be explicitly provided.

Can be specified as:

  • a command line option, e.g. --directory_registration_host=foo.bar,
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_DIRECTORY_REGISTRATION_HOST=foo.bar,
  • it has no default value (i.e. self determined IP address is used).

directory_registration_properties​

These are the properties that will be registered to the Directory for the Trial Datastore services. When inquiring the Directory, the properties inquired must match the properties registered. This is a string representing multiple properties in the form "name=value,name=value,name=value" where the values are optional.

Can be specified as:

  • a command line option, e.g. --directory_registration_properties="Sim=20,hpp,mem=HIGH",
  • an environment variable, e.g. COGMENT_TRIAL_DATASTORE_DIRECTORY_REGISTRATION_PROPERTIES="Sim=20,hpp,mem=HIGH",
  • it has no default value.
Last updated on