Concept of offset based metric store strategy in Cassandra

Related issue: #44

Current metric store strategy in Cassandra

Metrics are stored in Cassandra to a table created with following DDL:

CREATE TABLE metric.metric (
  tenant text,
  path text,
  time bigint,
  data double,
  PRIMARY KEY ((tenant, path), time)
) WITH CLUSTERING ORDER BY (time ASC)
  AND bloom_filter_fp_chance = 0.01
  AND caching = {'keys': 'ALL', 'rows_per_partition': '10'}
  AND comment = ''
  AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 'sstable_size_in_mb': '640'}
  AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
  AND crc_check_chance = 1.0
  AND dclocal_read_repair_chance = 0.0
  AND default_time_to_live = 0
  AND gc_grace_seconds = 864000
  AND max_index_interval = 2048
  AND memtable_flush_period_in_ms = 0
  AND min_index_interval = 128
  AND read_repair_chance = 0.0
  AND speculative_retry = 'NONE';

Let's assume that we want to store time series X which has data corresponding time range including 0 to 900 to this table. We have corresponding partition key X to time series X in Cassandra and all the data will be stored in a row as follows:

This will cause the following problems if we store more data for time series X as the time goes by:

• Data for X will not be distributed except for its replications. Not tolerable for X.
• Have a chance to have imbalance sized volumes between Cassandra nodes.
• Scanning cost for X in Cassandra will increase proposition to the size of the volume of data for time series X.

To avoid the troublesome situations, we will borrow the concept described in following project: CASSANDRA_DESIGN

Concept of offset based metric store strategy in Cassandra

We're going to introduce offset based store strategy in future version. Data will be stored in a table created with following DDL:

CREATE TABLE metric_offset.metric (
  tenant text,
  path text,
  startTime bigint,
  offset smallint,
  data double,
  PRIMARY KEY ((tenant, path, startTime), offset)
) WITH CLUSTERING ORDER BY (offset ASC)
  AND bloom_filter_fp_chance = 0.01
  AND caching = {'keys': 'ALL', 'rows_per_partition': '10'}
  AND comment = ''
  AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 'sstable_size_in_mb': '640'}
  AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
  AND crc_check_chance = 1.0
  AND dclocal_read_repair_chance = 0.0
  AND default_time_to_live = 0
  AND gc_grace_seconds = 864999
  AND max_index_interval = 2048
  AND memtable_flush_period_in_ms = 0
  AND min_index_interval = 128
  AND read_repair_chance = 0.0
  AND speculative_retry = 'NONE';

Let's again we assume the same situation in the above. We have to introduce a parameter bucketSize to be used to store data in this table. Let's assume the bucketSize is 300. Then data will be stored as follows:

The followings are different to the current version:

• startTime is used for partition key for X. startTime can be think of a bucket for data starting from startTime.
• offset will be used instead of time. We will use short to decrease size of each data point.

The expected benefits are:

• Data for time series X will be distributed over Cassandra nodes.
• Scanning cost will have upper limit because data is distributed over time buckets. This will decrease the latency when system reads data.
• Size of each data point will be decreased dramatically because we can represent each timestamp in short, not bigint which consumes 2 bytes per data point contrary to 8 bytes.

The expected troubles are:

• Have a chance to lose partial data.
• Have to make at least the same number of requests or more to the current when system tries to read data in certain time range because data is distributed over time buckets.