native_protocol_v5.html-diff

Erstellt Diff läuft nie ab
361 Entfernungen
638 Zeilen
207 Hinzufügungen
548 Zeilen
<!DOCTYPE html>
<!DOCTYPE html>
<html>
<html>
<head>
<head>
<title>CQL BINARY PROTOCOL v5</title>
<meta charset="utf-8">
<style>
<title>CQL BINARY PROTOCOL v5</title>
nav ol {
<style>
margin: 0;
nav ol { margin: 0; padding: 0; padding-left: 1em; }
padding: 0;
nav li { list-style: none; }
padding-left: 1em;
nav.top ul { margin: 0; padding: 0; background: #eee; color: black; }
}
nav.top ul li { display: inline-block; }
nav li {
</style>
list-style: none;
}
nav.top ul {
margin: 0;
padding: 0;
background: #eee;
color: black;
}
nav.top ul li {
display: inline-block;
}
</style>
</head>
</head>
<body>
<body>

<!-- -->
<h1>CQL BINARY PROTOCOL v5</h1>
<!-- Licensed to the Apache Software Foundation (ASF) under one -->
<h2>Table of Contents</h2>
<!-- or more contributor license agreements. See the NOTICE file -->
<nav>
<!-- distributed with this work for additional information -->
<ol>
<!-- regarding copyright ownership. The ASF licenses this file -->
<!-- to you under the Apache License, Version 2.0 (the -->
<li id="toc1">1
<!-- &quot;License&quot;); you may not use this file except in compliance -->
<a href="#s1">Overview</a>
<!-- with the License. You may obtain a copy of the License at -->
<!-- -->
</li>
<!-- http://www.apache.org/licenses/LICENSE-2.0 -->
<!-- -->
<li id="toc2">2
<!-- Unless required by applicable law or agreed to in writing, software -->
<a href="#s2">Frame format</a>
<!-- distributed under the License is distributed on an &quot;AS IS&quot; BASIS, -->
<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
<ol>
<!-- See the License for the specific language governing permissions and -->
<!-- limitations under the License. -->
<li id="toc2.1">2.1
<!-- -->
<h1>CQL BINARY PROTOCOL v5</h1>
<h2>Table of Contents</h2>
<nav>
<ol>
<li id="toc1">
1
<a href="#s1">Overview</a>
</li>
<li id="toc2">
2
<a href="#s2">Frame format</a>
<ol>
<li id="toc2.1">
2.1
<a href="#s2.1">Uncompressed Format</a>
<a href="#s2.1">Uncompressed Format</a>
</li>
</li>
<li id="toc2.2">
2.2
<li id="toc2.2">2.2
<a href="#s2.2">Compressed Format</a>
<a href="#s2.2">Compressed Format</a>
</li>
</li>
<li id="toc2.3">
2.3
<li id="toc2.3">2.3
<a href="#s2.3">Protocol Negotiation</a>
<a href="#s2.3">Protocol Negotiation</a>
<ol>
<ol>
<li id="toc2.3.1">
<li id="toc2.3.1">2.3.1
2.3.1
<a href="#s2.3.1">Initial Handshake</a>
<a href="#s2.3.1">Initial Handshake</a>
</li>
</li>
<li id="toc2.3.2">
2.3.2
<li id="toc2.3.2">2.3.2
<a href="#s2.3.2">Compression</a>
<a href="#s2.3.2">Compression</a>
</li>
</li>

</ol>
</ol>
</li>
</li>
<li id="toc2.4">
2.4
<li id="toc2.4">2.4
<a href="#s2.4">Frame Payload</a>
<a href="#s2.4">Frame Payload</a>
<ol>
<li id="toc2.4.1.1">2.4.1.1
<a href="#s2.4.1.1">version</a>
<ol>
<ol>
<li id="toc2.4.1.1">
<li id="toc2.4.1.2">2.4.1.2
2.4.1.1
<a href="#s2.4.1.2">flags</a>
<a href="#s2.4.1.1">version</a>
<ol>
</li>
<li id="toc2.4.1.2">
2.4.1.2
<li id="toc2.4.1.3">2.4.1.3
<a href="#s2.4.1.2">flags</a>
<a href="#s2.4.1.3">stream</a>
</li>
<li id="toc2.4.1.3">
</li>
2.4.1.3
<a href="#s2.4.1.3">stream</a>
<li id="toc2.4.1.4">2.4.1.4
</li>
<a href="#s2.4.1.4">opcode</a>
<li id="toc2.4.1.4">
2.4.1.4
</li>
<a href="#s2.4.1.4">opcode</a>
</li>
<li id="toc2.4.1.5">2.4.1.5
<li id="toc2.4.1.5">
<a href="#s2.4.1.5">length</a>
2.4.1.5
<a href="#s2.4.1.5">length</a>
</li>
</li>
</ol>

</li>
</ol>
</li>

</ol>
</li>

</ol>
</ol>
</li>
</li>
</ol>
</li>
<li id="toc3">3
<li id="toc3">
<a href="#s3">Notations</a>
3
<a href="#s3">Notations</a>
</li>
</li>
<li id="toc4">
<li id="toc4">4
4
<a href="#s4">Messages</a>
<a href="#s4">Messages</a>
<ol>
<ol>
<li id="toc4.1">
4.1
<li id="toc4.1">4.1
<a href="#s4.1">Requests</a>
<a href="#s4.1">Requests</a>
<ol>
<ol>
<li id="toc4.1.1">
<li id="toc4.1.1">4.1.1
4.1.1
<a href="#s4.1.1">STARTUP</a>
<a href="#s4.1.1">STARTUP</a>
</li>
</li>
<li id="toc4.1.2">
4.1.2
<li id="toc4.1.2">4.1.2
<a href="#s4.1.2">AUTH_RESPONSE</a>
<a href="#s4.1.2">AUTH_RESPONSE</a>
</li>
<li id="toc4.1.3">
</li>
4.1.3
<a href="#s4.1.3">OPTIONS</a>
<li id="toc4.1.3">4.1.3
</li>
<a href="#s4.1.3">OPTIONS</a>
<li id="toc4.1.4">
4.1.4
</li>
<a href="#s4.1.4">QUERY</a>
</li>
<li id="toc4.1.4">4.1.4
<li id="toc4.1.5">
<a href="#s4.1.4">QUERY</a>
4.1.5
<a href="#s4.1.5">PREPARE</a>
</li>
</li>
<li id="toc4.1.6">
<li id="toc4.1.5">4.1.5
4.1.6
<a href="#s4.1.5">PREPARE</a>
<a href="#s4.1.6">EXECUTE</a>
</li>
</li>
<li id="toc4.1.7">
4.1.7
<li id="toc4.1.6">4.1.6
<a href="#s4.1.7">BATCH</a>
<a href="#s4.1.6">EXECUTE</a>
</li>
<li id="toc4.1.8">
</li>
4.1.8
<a href="#s4.1.8">REGISTER</a>
<li id="toc4.1.7">4.1.7
</li>
<a href="#s4.1.7">BATCH</a>
</li>
<li id="toc4.1.8">4.1.8
<a href="#s4.1.8">REGISTER</a>
</li>

</ol>
</ol>
</li>
</li>
<li id="toc4.2">
4.2
<li id="toc4.2">4.2
<a href="#s4.2">Responses</a>
<a href="#s4.2">Responses</a>
<ol>
<li id="toc4.2.1">4.2.1
<a href="#s4.2.1">ERROR</a>
</li>
<li id="toc4.2.2">4.2.2
<a href="#s4.2.2">READY</a>
</li>
<li id="toc4.2.3">4.2.3
<a href="#s4.2.3">AUTHENTICATE</a>
</li>
<li id="toc4.2.4">4.2.4
<a href="#s4.2.4">SUPPORTED</a>
</li>
<li id="toc4.2.5">4.2.5
<a href="#s4.2.5">RESULT</a>
<ol>
<ol>
<li id="toc4.2.1">
<li id="toc4.2.5.1">4.2.5.1
4.2.1
<a href="#s4.2.5.1">Void</a>
<a href="#s4.2.1">ERROR</a>
</li>
</li>
<li id="toc4.2.2">
4.2.2
<li id="toc4.2.5.2">4.2.5.2
<a href="#s4.2.2">READY</a>
<a href="#s4.2.5.2">Rows</a>
</li>
<li id="toc4.2.3">
</li>
4.2.3
<a href="#s4.2.3">AUTHENTICATE</a>
<li id="toc4.2.5.3">4.2.5.3
</li>
<a href="#s4.2.5.3">Set_keyspace</a>
<li id="toc4.2.4">
4.2.4
</li>
<a href="#s4.2.4">SUPPORTED</a>
</li>
<li id="toc4.2.5.4">4.2.5.4
<li id="toc4.2.5">
<a href="#s4.2.5.4">Prepared</a>
4.2.5
<a href="#s4.2.5">RESULT</a>
</li>
<ol>
<li id="toc4.2.5.1">
<li id="toc4.2.5.5">4.2.5.5
4.2.5.1
<a href="#s4.2.5.5">Schema_change</a>
<a href="#s4.2.5.1">Void</a>
</li>
</li>
<li id="toc4.2.5.2">
4.2.5.2

<a href="#s4.2.5.2">Rows</a>
</ol>
</li>
<li id="toc4.2.5.3">
</li>
4.2.5.3
<a href="#s4.2.5.3">Set_keyspace</a>
<li id="toc4.2.6">4.2.6
</li>
<a href="#s4.2.6">EVENT</a>
<li id="toc4.2.5.4">
4.2.5.4
</li>
<a href="#s4.2.5.4">Prepared</a>
</li>
<li id="toc4.2.7">4.2.7
<li id="toc4.2.5.5">
<a href="#s4.2.7">AUTH_CHALLENGE</a>
4.2.5.5
<a href="#s4.2.5.5">Schema_change</a>
</li>
</li>
</ol>
<li id="toc4.2.8">4.2.8
</li>
<a href="#s4.2.8">AUTH_SUCCESS</a>
<li id="toc4.2.6">
4.2.6
</li>
<a href="#s4.2.6">EVENT</a>
</li>

<li id="toc4.2.7">
</ol>
4.2.7
<a href="#s4.2.7">AUTH_CHALLENGE</a>
</li>
</li>
<li id="toc4.2.8">

4.2.8
</ol>
<a href="#s4.2.8">AUTH_SUCCESS</a>
</li>
</li>
<li id="toc5">5
<a href="#s5">Data Type Serialization Formats</a>
</li>
<li id="toc6">6
<a href="#s6">User Defined Type Serialization</a>
</li>
<li id="toc7">7
<a href="#s7">Result paging</a>
</li>
<li id="toc8">8
<a href="#s8">Error codes</a>
</li>
<li id="toc9">9
<a href="#s9">Changes from v4</a>
</li>

</ol>
</ol>
</nav>
</li>
</ol>
<h2 id="s"> </h2>
</li>
<pre></pre>
<li id="toc5">
5
<h2 id="s1">1 Overview</h2>
<a href="#s5">Data Type Serialization Formats</a>
<pre> The CQL binary protocol is a frame based protocol with frames comprising a header, payload
</li>
<li id="toc6">
6
<a href="#s6">User Defined Type Serialization</a>
</li>
<li id="toc7">
7
<a href="#s7">Result paging</a>
</li>
<li id="toc8">
8
<a href="#s8">Error codes</a>
</li>
<li id="toc9">
9
<a href="#s9">Changes from v4</a>
</li>
</ol>
</nav>
<h2 id="s1">1 Overview</h2>
<pre> The CQL binary protocol is a frame based protocol with frames comprising a header, payload
and trailer. In v5 there are two distinct frame formats, compressed and uncompressed and in
and trailer. In v5 there are two distinct frame formats, compressed and uncompressed and in
both cases the payload is a stream of CQL envelopes (<a href="#s2.4">Section 2.4</a>). Each envelope contains
both cases the payload is a stream of CQL envelopes (<a href="#s2.4">Section 2.4</a>). Each envelope contains
a single CQL message, along with a metadata header. In effect, the framing format is a
a single CQL message, along with a metadata header. In effect, the framing format is a
simple wrapper around protocol v5.
simple wrapper around protocol v5.


In either format, a frame may or may not be self contained. If self contained, then the
In either format, a frame may or may not be self contained. If self contained, then the
payload includes one or more complete envelopes and can be fully processed immediately.
payload includes one or more complete envelopes and can be fully processed immediately.
Otherwise, the payload contains some part of a large envelope, which has been split into
Otherwise, the payload contains some part of a large envelope, which has been split into
its own sequence of frames. These are expected to be transmitted/received in order, so
its own sequence of frames. These are expected to be transmitted/received in order, so
the receiver can accumulate them as they arrive and process them once all have been received.
the receiver can accumulate them as they arrive and process them once all have been received.


The frame header contains length information for the payload, a flag to indicate whether
The frame header contains length information for the payload, a flag to indicate whether
or not the frame is self contained and a CRC24 to assert the integrity of the header itself.
or not the frame is self contained and a CRC24 to assert the integrity of the header itself.
There are slight variations in the header format between the compressed and uncompressed
There are slight variations in the header format between the compressed and uncompressed
variants.
variants.


The payload is opaque as far as the framing format is concerned, modulo the self
The payload is opaque as far as the framing format is concerned, modulo the self
contained variation.
contained variation.


The trailer contains a CRC32 to protect the integrity of the payload, covering all envelopes
The trailer contains a CRC32 to protect the integrity of the payload, covering all envelopes
(whole or partial) contained therein.
(whole or partial) contained therein.


All the integers in the v5 framing format are little-endian.
All the integers in the v5 framing format are little-endian.
</pre>
</pre>
<h2 id="s2">2 Frame Format</h2>
<h2 id="s2">2 Frame Format</h2>
<pre></pre>
<pre></pre>
<h3 id="s2.1">2.1 Uncompressed Format</h3>
<pre> The uncompressed variant uses a 6 byte header. The first 3 bytes contain payload length and
<h3 id="s2.1">2.1 Uncompressed Format</h3>
<pre> The uncompressed variant uses a 6 byte header. The first 3 bytes contain payload length and
self contained flag. The following 3 bytes contain the CRC24 for the header itself.
self contained flag. The following 3 bytes contain the CRC24 for the header itself.
The max size for the payload is 128KiB, and is followed by its CRC32.
The max size for the payload is 128KiB, and is followed by its CRC32.


The first 3 bytes of the header represent a single unsigned little-endian integer. The self
The first 3 bytes of the header represent a single unsigned little-endian integer. The self
contained flag is the 17th bit of that decoded integer.
contained flag is the 17th bit of that decoded integer.


1. Payload length (17 bits)
1. Payload length (17 bits)
2. isSelfContained flag (1 bit)
2. isSelfContained flag (1 bit)
3. Header padding (6 bits)
3. Header padding (6 bits)
4. CRC24 of the header (24 bits)
4. CRC24 of the header (24 bits)
5. Payload (up to 2 ^ 17 - 1 bytes)
5. Payload (up to 2 ^ 17 - 1 bytes)
6. Payload CRC32 (32 bits)
6. Payload CRC32 (32 bits)


0 1 2 3
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length |C| |
| Payload Length |C| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
CRC24 of Header | |
CRC24 of Header | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
| |
| |
+ +
+ +
| Payload |
| Payload |
+ +
+ +
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC32 of Payload |
| CRC32 of Payload |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


The headed padding can be discarded.
The headed padding can be discarded.


The CRC24 of the header is calculated by initializing the CRC24 algorithm
The CRC24 of the header is calculated by initializing the CRC24 algorithm
with the value 0x875060 and using the polynomial 0x1974F0B.
with the value 0x875060 and using the polynomial 0x1974F0B.
</pre>
</pre>
<h3 id="s2.2">2.2 LZ4 Compressed Format</h3>
<h3 id="s2.2">2.2 LZ4 Compressed Format</h3>
<pre> The variant with LZ4 compression uses an 8 byte header. The first 4 bytes contain
<pre> The variant with LZ4 compression uses an 8 byte header. The first 4 bytes contain
an unsigned integer (little endian) containing the compressed and uncompressed
an unsigned integer (little endian) containing the compressed and uncompressed
lengths of the payload and the self contained flag. The remaining 4 bytes are an
lengths of the payload and the self contained flag. The remaining 4 bytes are an
unsigned integer (little endian) representing the CRC24 for the header.
unsigned integer (little endian) representing the CRC24 for the header.
As with uncompressed frames, the max payload size is 128KiB and is followed
As with uncompressed frames, the max payload size is 128KiB and is followed
by a CRC32 trailer. This is the CRC of the compressed payload.
by a CRC32 trailer. This is the CRC of the compressed payload.


1. Compressed length (17 bits)
1. Compressed length (17 bits)
2. Uncompressed length (17 bits)
2. Uncompressed length (17 bits)
3. isSelfContained flag (1 bit)
3. isSelfContained flag (1 bit)
4. Header padding (5 bits)
4. Header padding (5 bits)
5. CRC24 of Header contents (24 bits)
5. CRC24 of Header contents (24 bits)
6. Compressed Payload (up to 2 ^ 17 - 1 bits)
6. Compressed Payload (up to 2 ^ 17 - 1 bits)
7. CRC32 of Compressed Payload (32 bits)
7. CRC32 of Compressed Payload (32 bits)


0 1 2 3
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compressed Length | Uncompressed Length
| Compressed Length | Uncompressed Length
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C| | CRC24 of Header |
|C| | CRC24 of Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| |
+ +
+ +
| Compressed Payload |
| Compressed Payload |
+ +
+ +
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC32 of Compressed Payload |
| CRC32 of Compressed Payload |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


The header padding can be discarded.
The header padding can be discarded.


An uncompressed length of 0 signals that the compressed payload
An uncompressed length of 0 signals that the compressed payload
should be used as-is and not decompressed.
should be used as-is and not decompressed.


The CRC24 of the header is calculated with the same parameters as for
The CRC24 of the header is calculated with the same parameters as for
the uncompressed format (<a href="#s2.1">Section 2.1</a>).
the uncompressed format (<a href="#s2.1">Section 2.1</a>).
</pre>
</pre>
<h3 id="s2.3">2.3 Protocol Negotiation</h3>
<h3 id="s2.3">2.3 Protocol Negotiation</h3>
<pre></pre>
<pre></pre>
<h4 id="s2.3.1">2.3.1 Initial Handshake</h4>
<pre> In order to support both v5 and earlier formats, the v5 framing format is not
<h4 id="s2.3.1">2.3.1 Initial Handshake</h4>
<pre> In order to support both v5 and earlier formats, the v5 framing format is not
applied to message exchanges before an initial handshake is completed. Practically,
applied to message exchanges before an initial handshake is completed. Practically,
this means that the initial STARTUP message and any OPTIONS messages which precede
this means that the initial STARTUP message and any OPTIONS messages which precede
it are expected to be unframed. Likewise, the responses returned by the server,
it are expected to be unframed. Likewise, the responses returned by the server,
SUPPORTED in response to OPTIONS and either READY or AUTHENTICATE in response to
SUPPORTED in response to OPTIONS and either READY or AUTHENTICATE in response to
STARTUP are transmitted unframed.
STARTUP are transmitted unframed.


After sending the READY or AUTHENTICATE response to a STARTUP message, the server
After sending the READY or AUTHENTICATE response to a STARTUP message, the server
will begin encoding and decoding all further transmissions according to the protocol
will begin encoding and decoding all further transmissions according to the protocol
version of that STARTUP message. Compression of the frames is dictated by the
version of that STARTUP message. Compression of the frames is dictated by the
COMPRESSION option sent in the STARTUP message. Only LZ4 compression is currently
COMPRESSION option sent in the STARTUP message. Only LZ4 compression is currently
supported for v5.
supported for v5.


Note: OPTIONS requests may be sent by the client at any time in the connection
Note: OPTIONS requests may be sent by the client at any time in the connection
lifecycle, both before and after the STARTUP exchange. As mentioned, those
lifecycle, both before and after the STARTUP exchange. As mentioned, those
transmitted before STARTUP, as well as the SUPPORTED responses the server returns
transmitted before STARTUP, as well as the SUPPORTED responses the server returns
are unframed. Any OPTIONS/SUPPORTED exchanges after the STARTUP handshake are
are unframed. Any OPTIONS/SUPPORTED exchanges after the STARTUP handshake are
formatted according to the negotiated protocol version, so for v5 these must be
formatted according to the negotiated protocol version, so for v5 these must be
framed.
framed.
</pre>
</pre>
<h4 id="s2.3.2">2.3.2 Compression</h4>
<h4 id="s2.3.2">2.3.2 Compression</h4>
<pre> Before being used, client and server must agree on a compression algorithm to
<pre> Before being used, client and server must agree on a compression algorithm to
use, which is done in the STARTUP message. As a consequence, a STARTUP message
use, which is done in the STARTUP message. As a consequence, a STARTUP message
must never be compressed. However, once the STARTUP frame has been received
must never be compressed. However, once the STARTUP frame has been received
by the server, messages can be compressed (including the response to the STARTUP
by the server, messages can be compressed (including the response to the STARTUP
request). Frames do not have to be compressed, however, even if compression has
request). Frames do not have to be compressed, however, even if compression has
been agreed upon (a sender may only compress frames above a certain size at its
been agreed upon (a sender may only compress frames above a certain size at its
discretion). Where compression has been agreed, the sender signals that the payload
discretion). Where compression has been agreed, the sender signals that the payload
is not compressed by setting the compressed length to 0.
is not compressed by setting the compressed length to 0.


As of v5 of the protocol, the only compression available is lz4
As of v5 of the protocol, the only compression available is lz4
(<a href="https://code.google.com/p/lz4/">https://code.google.com/p/lz4/</a>).
(<a href="https://code.google.com/p/lz4/">https://code.google.com/p/lz4/</a>).


</pre>
</pre>
<h3 id="s2.4">2.4 Frame Payload</h3>
<h3 id="s2.4">2.4 Frame Payload</h3>
<pre> Envelopes are defined as:
<pre> Envelopes are defined as:


0 8 16 24 32 40
0 8 16 24 32 40
+---------+---------+---------+---------+---------+
+---------+---------+---------+---------+---------+
| version | flags | stream | opcode |
| version | flags | stream | opcode |
+---------+---------+---------+---------+---------+
+---------+---------+---------+---------+---------+
| length |
| length |
+---------+---------+---------+---------+
+---------+---------+---------+---------+
| |
| |
. ... body ... .
. ... body ... .
. .
. .
. .
. .
+----------------------------------------
+----------------------------------------


All values in an envelope are big-endian (network byte order).
All values in an envelope are big-endian (network byte order).


Each envelope contains a fixed size header (9 bytes) followed by a variable size
Each envelope contains a fixed size header (9 bytes) followed by a variable size
body. The content of the body depends
body. The content of the body depends
on the header opcode value (the body can in particular be empty for some
on the header opcode value (the body can in particular be empty for some
opcode values). The list of allowed opcodes is defined in <a href="#s2.4.1.4">Section 2.4.1.4</a> and the
opcode values). The list of allowed opcodes is defined in <a href="#s2.4.1.4">Section 2.4.1.4</a> and the
details of each corresponding message are described <a href="#s4">Section 4</a>.
details of each corresponding message are described <a href="#s4">Section 4</a>.


The protocol distinguishes two types of envelope: requests and responses. Requests
The protocol distinguishes two types of envelope: requests and responses. Requests
are those envelopes sent by the client to the server. Responses are those envelopes sent
are those envelopes sent by the client to the server. Responses are those envelopes sent
by the server to the client. Note, however, that the protocol supports server pushes
by the server to the client. Note, however, that the protocol supports server pushes
(events) so a response does not necessarily come right after a client request.
(events) so a response does not necessarily come right after a client request.


Note to client implementors: client libraries should always assume that the
Note to client implementors: client libraries should always assume that the
body of a given envelope may contain more data than what is described in this
body of a given envelope may contain more data than what is described in this
document. It will however always be safe to ignore the remainder of the body
document. It will however always be safe to ignore the remainder of the body
in such cases. The reason is that this may enable extending the protocol
in such cases. The reason is that this may enable extending the protocol
with optional features without needing to change the protocol version.
with optional features without needing to change the protocol version.


Envelope headers are designed to support backwards compatibility with earlier
Envelope headers are designed to support backwards compatibility with earlier
protocol versions. For that reason, they include an unused leading byte in place
protocol versions. For that reason, they include an unused leading byte in place
of the version field from previous protocol versions. This was always to some extent
of the version field from previous protocol versions. This was always to some extent
redundant as the version is set and enforced at the connection level. It was also
redundant as the version is set and enforced at the connection level. It was also
previously possible to enable compression for an individual envelope. This is no
previously possible to enable compression for an individual envelope. This is no
longer possible, as the framing format is responsible for compression, which is set for
longer possible, as the framing format is responsible for compression, which is set for
the lifetime of a connection and applies to all messages transmitted throughout it
the lifetime of a connection and applies to all messages transmitted throughout it
(see <a href="#s2.2">Section 2.2</a> for caveats). The compression flag is therefore deprecated and
(see <a href="#s2.2">Section 2.2</a> for caveats). The compression flag is therefore deprecated and
ignored in protocol v5.
ignored in protocol v5.
</pre>
</pre>
<h5 id="s2.4.1.1">2.4.1.1 version</h5>
<h5 id="s2.4.1.1">2.4.1.1 version</h5>
<pre> The version is a single byte that indicates both the direction of the message
<pre> The version is a single byte that indicates both the direction of the message
(request or response) and the version of the protocol in use. The most
(request or response) and the version of the protocol in use. The most
significant bit of version is used to define the direction of the message:
significant bit of version is used to define the direction of the message:
0 indicates a request, 1 indicates a response. This can be useful for protocol
0 indicates a request, 1 indicates a response. This can be useful for protocol
analyzers to distinguish the nature of the packet from the direction in which
analyzers to distinguish the nature of the packet from the direction in which
it is moving. The rest of that byte is the protocol version (5 for the protocol
it is moving. The rest of that byte is the protocol version (5 for the protocol
defined in this document). In other words, for this version of the protocol,
defined in this document). In other words, for this version of the protocol,
version will be one of:
version will be one of:
0x05 Request frame for this protocol version
0x05 Request frame for this protocol version
0x85 Response frame for this protocol version
0x85 Response frame for this protocol version


Please note that while every message ships with the version, only one version
Please note that while every message ships with the version, only one version
of messages is accepted on a given connection. In other words, the first message
of messages is accepted on a given connection. In other words, the first message
exchanged (STARTUP) sets the version for the connection for the lifetime of this
exchanged (STARTUP) sets the version for the connection for the lifetime of this
connection.
connection.


This document describes version 5 of the protocol. For the changes made since
This document describes version 5 of the protocol. For the changes made since
version 4, see <a href="#s9">Section 9</a>.
version 4, see <a href="#s9">Section 9</a>.
</pre>
</pre>
<h5 id="s2.4.1.2">2.4.1.2 flags</h5>
<h5 id="s2.4.1.2">2.4.1.2 flags</h5>
<pre> Flags applying to this envelope. The flags have the following meaning (described
<pre> Flags applying to this envelope. The flags have the following meaning (described
by the mask that allows selecting them):
by the mask that allows selecting them):
0x01: Compression flag. In protocol v5 this flag is deprecated and ignored.
0x01: Compression flag. In protocol v5 this flag is deprecated and ignored.
0x02: Tracing flag. For a request, this indicates the client requires tracing
0x02: Tracing flag. For a request, this indicates the client requires tracing
of the request. Note that only QUERY, PREPARE and EXECUTE queries
of the request. Note that only QUERY, PREPARE and EXECUTE queries
support tracing. Other requests will simply ignore the tracing flag if
support tracing. Other requests will simply ignore the tracing flag if
set. If a request supports tracing and the tracing flag is set, the
set. If a request supports tracing and the tracing flag is set, the
response to this request will have the tracing flag set and contain
response to this request will have the tracing flag set and contain
tracing information.
tracing information.
If a response has the tracing flag set, its body contains a tracing ID.
If a response has the tracing flag set, its body contains a tracing ID.
The tracing ID is a [uuid] and is the first thing in the body.
The tracing ID is a [uuid] and is the first thing in the body.
0x04: Custom payload flag. For a request or response, this indicates that a
0x04: Custom payload flag. For a request or response, this indicates that a
generic key-value custom payload for a custom QueryHandler implementation
generic key-value custom payload for a custom QueryHandler implementation
is present. Such a custom payload is simply ignored by the default
is present. Such a custom payload is simply ignored by the default
QueryHandler implementation. Currently, only QUERY, PREPARE, EXECUTE and
QueryHandler implementation. Currently, only QUERY, PREPARE, EXECUTE and
BATCH requests support custom payloads.
BATCH requests support custom payloads.
Type of custom payload is [bytes map] (see below). If either or both
Type of custom payload is [bytes map] (see below). If either or both
of the tracing and warning flags are set, the custom payload will follow
of the tracing and warning flags are set, the custom payload will follow
those indicated elements in the body. If neither are set, the custom
those indicated elements in the body. If neither are set, the custom
payload will be the first value in the body.
payload will be the first value in the body.
0x08: Warning flag. The response contains warnings which were generated by the
0x08: Warning flag. The response contains warnings which were generated by the
server to go along with this response.
server to go along with this response.
If a response has the warning flag set, its body will contain the text of
If a response has the warning flag set, its body will contain the text of
the warnings. The warnings are a [string list] and will be the first value
the warnings. The warnings are a [string list] and will be the first value
in the body if the tracing flag is not set, or directly after the tracing
in the body if the tracing flag is not set, or directly after the tracing
ID if it is.
ID if it is.
0x10: Use beta flag. Indicates that the client opts in to use protocol version
0x10: Use beta flag. Indicates that the client opts in to use protocol version
that is currently in beta. Server will respond with ERROR if protocol
that is currently in beta. Server will respond with ERROR if protocol
version is marked as beta on server and client does not provide this flag.
version is marked as beta on server and client does not provide this flag.


The rest of flags is currently unused and ignored.
The rest of flags is currently unused and ignored.
</pre>
</pre>
<h5 id="s2.4.1.3">2.4.1.3 stream</h5>
<h5 id="s2.4.1.3">2.4.1.3 stream</h5>
<pre> An envelope has a stream id (a [short] value). When sending request messages, this
<pre> An envelope has a stream id (a [short] value). When sending request messages, this
stream id must be set by the client to a non-negative value (negative stream id
stream id must be set by the client to a non-negative value (negative stream id
are reserved for streams initiated by the server; currently all EVENT messages
are reserved for streams initiated by the server; currently all EVENT messages
(<a href="#s4.2.6">section 4.2.6</a>) have a streamId of -1). If a client sends a request message
(<a href="#s4.2.6">section 4.2.6</a>) have a streamId of -1). If a client sends a request message
with the stream id X, it is guaranteed that the stream id of the response to
with the stream id X, it is guaranteed that the stream id of the response to
that message will be X.
that message will be X.


This helps to enable the asynchronous nature of the protocol. If a client
This helps to enable the asynchronous nature of the protocol. If a client
sends multiple messages simultaneously (without waiting for responses), there
sends multiple messages simultaneously (without waiting for responses), there
is no guarantee on the order of the responses. For instance, if the client
is no guarantee on the order of the responses. For instance, if the client
writes REQ_1, REQ_2, REQ_3 on the wire (in that order), the server might
writes REQ_1, REQ_2, REQ_3 on the wire (in that order), the server might
respond to REQ_3 (or REQ_2) first. Assigning different stream ids to these 3
respond to REQ_3 (or REQ_2) first. Assigning different stream ids to these 3
requests allows the client to distinguish to which request a received answer
requests allows the client to distinguish to which request a received answer
responds to. As there can only be 32768 different simultaneous streams, it is up
responds to. As there can only be 32768 different simultaneous streams, it is up
to the client to reuse stream id.
to the client to reuse stream id.


Note that clients are free to use the protocol synchronously (i.e. wait for
Note that clients are free to use the protocol synchronously (i.e. wait for
the response to REQ_N before sending REQ_N+1). In that case, the stream id
the response to REQ_N before sending REQ_N+1). In that case, the stream id
can be safely set to 0 as long as each frame contains only a single envelope.
can be safely set to 0 as long as each frame contains only a single envelope.
Clients should also feel free to use only a subset of the 32768 maximum possible stream
Clients should also feel free to use only a subset of the 32768 maximum possible stream
ids if it is simpler for its implementation.
ids if it is simpler for its implementation.
</pre>
</pre>
<h5 id="s2.4.1.4">2.4.1.4 opcode</h5>
<h5 id="s2.4.1.4">2.4.1.4 opcode</h5>
<pre> An integer byte that distinguishes the actual message:
<pre> An integer byte that distinguishes the actual message:
0x00 ERROR
0x00 ERROR
0x01 STARTUP
0x01 STARTUP
0x02 READY
0x02 READY
0x03 AUTHENTICATE
0x03 AUTHENTICATE
0x05 OPTIONS
0x05 OPTIONS
0x06 SUPPORTED
0x06 SUPPORTED
0x07 QUERY
0x07 QUERY
0x08 RESULT
0x08 RESULT
0x09 PREPARE
0x09 PREPARE
0x0A EXECUTE
0x0A EXECUTE
0x0B REGISTER
0x0B REGISTER
0x0C EVENT
0x0C EVENT
0x0D BATCH
0x0D BATCH
0x0E AUTH_CHALLENGE
0x0E AUTH_CHALLENGE
0x0F AUTH_RESPONSE
0x0F AUTH_RESPONSE
0x10 AUTH_SUCCESS
0x10 AUTH_SUCCESS


Messages are described in <a href="#s4">Section 4</a>.
Messages are described in <a href="#s4">Section 4</a>.


(Note that there is no 0x04 message in this version of the protocol)
(Note that there is no 0x04 message in this version of the protocol)
</pre>
</pre>
<h5 id="s2.4.1.5">2.4.1.5 length</h5>
<h5 id="s2.4.1.5">2.4.1.5 length</h5>
<pre> A 4 byte integer representing the length of the body of the envelope (note:
<pre> A 4 byte integer representing the length of the body of the envelope (note:
currently an envelope body is limited to 256MB in length).
currently an envelope body is limited to 256MB in length).


</pre>
</pre>
<h2 id="s3">3 Notations</h2>
<h2 id="s3">3 Notations</h2>
<pre> To describe the layout of the envelope body for the messages in <a href="#s4">Section 4</a>, we
<pre> To describe the layout of the envelope body for the messages in <a href="#s4">Section 4</a>, we
define the following:
define the following:


[int] A 4 bytes integer
[int] A 4 bytes integer
[long] A 8 bytes integer
[long] A 8 bytes integer
[byte] A 1 byte unsigned integer
[byte] A 1 byte unsigned integer
[short] A 2 bytes unsigned integer
[short] A 2 bytes unsigned integer
[string] A [short] n, followed by n bytes representing an UTF-8
[string] A [short] n, followed by n bytes representing an UTF-8
string.
string.
[long string] An [int] n, followed by n bytes representing an UTF-8 string.
[long string] An [int] n, followed by n bytes representing an UTF-8 string.
[uuid] A 16 bytes long uuid.
[uuid] A 16 bytes long uuid.
[string list] A [short] n, followed by n [string].
[string list] A [short] n, followed by n [string].
[bytes] A [int] n, followed by n bytes if n &gt;= 0. If n &lt; 0,
[bytes] A [int] n, followed by n bytes if n &gt;= 0. If n &lt; 0,
no byte should follow and the value represented is `null`.
no byte should follow and the value represented is `null`.
[value] A [int] n, followed by n bytes if n &gt;= 0.
[value] A [int] n, followed by n bytes if n &gt;= 0.
If n == -1 no byte should follow and the value represented is `null`.
If n == -1 no byte should follow and the value represented is `null`.
If n == -2 no byte should follow and the value represented is
If n == -2 no byte should follow and the value represented is
`not set` not resulting in any change to the existing value.
`not set` not resulting in any change to the existing value.
n &lt; -2 is an invalid value and results in an error.
n &lt; -2 is an invalid value and results in an error.
[short bytes] A [short] n, followed by n bytes if n &gt;= 0.
[short bytes] A [short] n, followed by n bytes if n &gt;= 0.


[unsigned vint] An unsigned variable length integer. A vint is encoded with the most significant byte (MSB) first.
[unsigned vint] An unsigned variable length integer. A vint is encoded with the most significant byte (MSB) first.
The most significant byte will contains the information about how many extra bytes need to be read
The most significant byte will contains the information about how many extra bytes need to be read
as well as the most significant bits of the integer.
as well as the most significant bits of the integer.
The number of extra bytes to read is encoded as 1 bits on the left side.
The number of extra bytes to read is encoded as 1 bits on the left side.
For example, if we need to read 2 more bytes the first byte will start with 110
For example, if we need to read 2 more bytes the first byte will start with 110
(e.g. 256 000 will be encoded on 3 bytes as [110]00011 11101000 00000000)
(e.g. 256 000 will be encoded on 3 bytes as [110]00011 11101000 00000000)
If the encoded integer is 8 bytes long the vint will be encoded on 9 bytes and the first
If the encoded integer is 8 bytes long the vint will be encoded on 9 bytes and the first
byte will be: 11111111
byte will be: 11111111


[vint] A signed variable length integer. This is encoded using zig-zag encoding and then sent
[vint] A signed variable length integer. This is encoded using zig-zag encoding and then sent
like an [unsigned vint]. Zig-zag encoding converts numbers as follows:
like an [unsigned vint]. Zig-zag encoding converts numbers as follows:
0 = 0, -1 = 1, 1 = 2, -2 = 3, 2 = 4, -3 = 5, 3 = 6 and so forth.
0 = 0, -1 = 1, 1 = 2, -2 = 3, 2 = 4, -3 = 5, 3 = 6 and so forth.
The purpose is to send small negative values as small unsigned values, so that we save bytes on the wire.
To encode a value n use &#34;(n &gt;&gt; 31) ^ (n &lt;&lt; 1)&#34; for 32 bit values, and &#34;(n &gt;&gt; 63) ^ (n &lt;&lt; 1)&#34;
for 64 bit values where &#34;^&#34; is the xor operation, &#34;&lt;&lt;&#34; is the left shift operation and &#34;&gt;&gt;&#34; is
the arithmetic right shift operation (highest-order bit is replicated).
Decode with &#34;(n &gt;&gt; 1) ^ -(n &amp; 1)&#34;.

[option] A pair of &lt;id&gt;&lt;value&gt; where &lt;id&gt; is a [short] representing
the option id and &lt;value&gt; depends on that option (and can be
of size 0). The supported id (and the corresp