Skip to content

Snowflake

Jump to bottom
Aaron Graubert edited this page May 15, 2019 · 2 revisions

SnowflakeClient

Hound's API is dependent on the fact that each database entry has a unique, and roughly time-sortable filename.

The SnowflakeClient is based on Twitter's Snowflake system. Snowflakes allow for safe concurrent access, since no two records will ever have the same name (preventing accidental overwrites) and saves time by allowing Hound to sort records by filename.

Note: The preferred way to construct a new SnowflakeClient is via hound.getclient()

API

  • hound.snowflake.SnowflakeClient(name): (Constructor)

    Constructs a new SnowflakeClient. The name is not important, but is used to help the user distinguish between multiple clients. The preferred way to construct a new SnowflakeClient is via hound.getclient()

  • hound.snowflake.SnowflakeClient(): (Call operator)

  • hound.snowflake.SnowflakeClient.snowflake():

    Generates a new 22-byte snowflake. Generally, this is a very fast operation, however, in the event of a system clock rollback, the client will block until after the timestamp used in the last snowflake it generated. See the spec below for the format of snowflakes

  • hound.snowflake.SnowflakeClient.unpack(snowflake, validate=True):

    Parses a given snowflake (either as a 22-byte bytes object, or a 44-byte hex string) into a hound.Snowflake named tuple. If validate is True, this will raise a ValueError if the snowflake has an invalid checksum

Snowflake Spec

Each snowflake consists of 22 bytes in big-endian order:

  • 8-byte unix timestamp as a floating point number
  • 8-byte machine id. This is based on the hardware's nodename, which is usually only 6-bytes, but some newer specifications use 8
  • 2-byte random client id. This is generated during the constructor of each client. This helps avoid conflicts between clients on the same machine
  • 2-byte sequence id. This starts at 0 (0x0000) and increments each time a snowflake is generated
  • 1-byte zero field. This is reserved for potential future use, but currently byte 21 will always be 0
  • 1-byte checksum field. The checksum is not directly used by hound, but may be useful in other systems

Clone this wiki locally