Snappier

Data stored per entry in lookup table: Range Bits-used Description ------------------------------------ 1..64 0..7 Literal/copy length encoded in opcode byte 0..7 8..10 Copy offset encoded in opcode byte / 256 0..4 11..13 Extra bytes after opcode We use eight bits for the length even though 7 would have sufficed because of efficiency reasons: (1) Extracting a byte is faster than a bit-field (2) It properly aligns copy offset so we do not need a <<8

This is a table of shuffle control masks that can be used as the source operand for PSHUFB to permute the contents of the destination XMM register into a repeating byte pattern.

j * (16 / j) for all j from 0 to 7. 0 is not actually used.

Copy [src, src+(opEnd-op)) to [op, (opEnd-op)) but faster than IncrementalCopySlow. buf_limit is the address past the end of the writable region of the buffer. May write past opEnd, but won't write past bufferEnd.

Pointer to the source point in the buffer. Pointer to the destination point in the buffer. Pointer to the end of the area to write in the buffer. Pointer past the end of the buffer. Fixed pointer to . Fixing the PshufbFillPatterns array for use in the SSSE3 optimized route is expensive, so we do that in the outer loop in and pass the pointer to this method. This makes the logic a bit more confusing, but is a significant performance boost.

A fake owner wrapping an empty .

Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0.

Return floor(log2(n)) for positive integer n.

Finds the index of the least significant non-zero bit.

Wraps an inner to have a shorter length.

Find the largest n such that s1[0,n-1] == s2[0,n-1] and n <= (s2_limit - s2). Return (n, n < 8). Does not read *s2_limit or beyond. Does not read *(s1 + (s2_limit - s2)) or beyond. Requires that s2_limit >= s2. In addition populate *data with the next 5 bytes from the end of the match. This is only done if 8 bytes are available (s2_limit - s2 >= 8). The point is that on some arch's this can be done faster in this routine than subsequent loading from s2 + n.

Decompress a portion of the input.

Input to process. Number of bytes processed from the input. The first call to this method after construction or after a call to start at the beginning of a new Snappy block, leading with the encoded block size. It may be called multiple times as more data becomes available. will be true once the entire block has been processed.

Read the uncompressed length stored at the start of the compressed data.

Input data, which should begin with the varint encoded uncompressed length. The length of the compressed data, or null if the length is not yet complete. This variant is used when reading a stream, and will pause if there aren't enough bytes available in the input. Subsequent calls with more data will resume processing.

Read the uncompressed length stored at the start of the compressed data.

Input data, which should begin with the varint encoded uncompressed length. The length of the uncompressed data. Invalid stream length

Throws an . This is in a separate subroutine to allow the calling subroutine to be inlined.

Exception message.

Extracts the data from from the block, returning a block of memory and resetting the block.

An block of memory. Caller is responsible for disposing. This provides a more efficient way to decompress an entire block in scenarios where the caller wants an owned block of memory and isn't going to reuse the SnappyDecompressor. It avoids the need to copy a block of memory calling .

Load some data into the output buffer, only used for testing.

Load a byte array into _scratch, only used for testing.

Only used for testing.

Emits the stream format used for Snappy streams.

Processes some input, potentially returning compressed data. Flush must be called when input is complete to get any remaining compressed data.

Uncompressed data to emit. Output stream. A block of memory with compressed data (if any). Must be used before any subsequent call to Write.

Processes some input, potentially returning compressed data. Flush must be called when input is complete to get any remaining compressed data.

Uncompressed data to emit. Output stream. Cancellation token. A block of memory with compressed data (if any). Must be used before any subsequent call to Write.

Processes up to one entire block from the input, potentially combining with previous input blocks. Fills the compressed data to the output buffer. Will not process more than one output block at a time to avoid overflowing the output buffer.

Input to compress. Number of bytes consumed.

Parses the stream format used for Snappy streams.

Assuming that we're at the beginning of a chunk, reads the CRC. If partially read, stores the value in _scratch for subsequent reads. Should not be called if chunkByteProcessed >= 4.

Routines for performing Snappy compression and decompression on raw data blocks using . These routines do not read or write any Snappy framing.

For a given amount of input data, calculate the maximum potential size of the compressed output.

Length of the input data, in bytes. The maximum potential size of the compressed output. This is useful for allocating a sufficient output buffer before calling .

Compress a block of Snappy data.

Data to compress. Buffer to receive the compressed data. Number of bytes written to . The output buffer must be large enough to contain the compressed output.

Compress a block of Snappy data.

Data to compress. An with the decompressed data. The caller is responsible for disposing this object. Failing to dispose of the returned may result in memory leaks.

Compress a block of Snappy data.

Data to compress. The resulting byte array is allocated on the heap. If possible, should be used instead since it uses a shared buffer pool.

Get the uncompressed data length from a compressed Snappy block.

Compressed snappy block. The length of the uncompressed data in the block. The data in has an invalid length. This is useful for allocating a sufficient output buffer before calling .

Decompress a block of Snappy data. This must be an entire block.

Data to decompress. Buffer to receive the decompressed data. Number of bytes written to . Invalid Snappy block. Output buffer is too small.

Decompress a block of Snappy to a new memory buffer. This must be an entire block.

Data to decompress. An with the decompressed data. The caller is responsible for disposing this object. Failing to dispose of the returned may result in memory leaks.

Decompress a block of Snappy to a new byte array. This must be an entire block.

Data to decompress. The decompressed data. The resulting byte array is allocated on the heap. If possible, should be used instead since it uses a shared buffer pool.

Stream which supports compressing or decompressing data using the Snappy compression algorithm. To decompress data, supply a stream to be read. To compress data, provide a stream to be written to.

Create a stream which supports compressing or decompressing data using the Snappy compression algorithm. To decompress data, supply a stream to be read. To compress data, provide a stream to be written to.

Source or destination stream. Compression or decompression mode. is null. Stream read/write capability doesn't match with . Invalid . The stream will be closed when the SnappyStream is closed.

Create a stream which supports compressing or decompressing data using the Snappy compression algorithm. To decompress data, supply a stream to be read. To compress data, provide a stream to be written to.

Source or destination stream. Compression or decompression mode. If true, close the stream when the SnappyStream is closed. is null. Stream read/write capability doesn't match with . Invalid .

The base stream being read from or written to.