Manual section: | 7 |
---|
This document describes the syntax used by Varnish Test Cases files (.vtc). A vtc file describe a scenario with different scripted HTTP-talking entities, and generally one or more Varnish instances to test.
A vtc file will be read word after word, with very little tokenization, meaning a syntax error won't be detected until the test actually reach the relevant action in the test.
A parsing error will most of the time result in an assert being triggered. If this happens, please refer yourself to the related source file and line number. However, this guide should help you avoid the most common mistakes.
The parser splits words by detecting whitespace characters and a string is a word, or a series of words on the same line enclosed by double-quotes ("..."), or, for multi-line strings, enclosed in curly brackets ({...}).
The leading whitespaces of lines are ignored. Empty lines (or ones consisting only of whitespaces) are ignored too, as are the lines starting with "#" that are comments.
Test files take at most one command per line, with the first word of the line being the command and the following ones being its arguments. To continue over to a new line without breaking the argument string, you can escape the newline character (n) with a backslash ().
NOTE: this can be used from the top-level as well as from client and server specifications.
Barriers allows you to synchronize different threads to make sure events occur in the right order. It's even possible to use them in VCL.
First, it's necessary to declare the barrier:
barrier bNAME TYPE NUMBER [-cyclic]
With the arguments being:
Then, to add a sync point:
barrier bNAME sync
This will block the parent thread until the number of sync points for bNAME reaches the NUMBER given in the barrier declaration.
If you wish to synchronize the VCL, you need to declare a "sock" barrier. This will emit a macro definition named "bNAME_sock" that you can use in VCL (after importing the debug vmod):
debug.barrier_sync("${bNAME_sock}");
This function returns 0 if everything went well and is the equivalent of barrier bNAME sync at the VTC top-level.
Client and server threads are fake HTTP entities used to test your Varnish and VCL. They take any number of arguments, and the one that are not recognized, assuming they don't start with '-', are treated as specifications, laying out the actions to undertake:
client cNAME [...]
server sNAME [...]
Clients and server are identified by a string that's the first argument, clients' names start with 'c' and servers' names start with 's'.
As the client and server commands share a good deal of arguments and specification actions, they are grouped in this single section, specific items will be explicitly marked as such.
Normally, to keep things simple, server threads only handle one connection at a time, but the -dispatch switch allows to accept any number of connection and handle them following the given spec.
However, -dispatch is only allowed for the server name "s0".
To make things easier in the general case, clients will connect by default to the first Varnish server declared and the -vcl+backend switch of the varnish command will add all the declared servers as backends.
Be careful though, servers will by default listen to the 127.0.0.1 IP and will pick a random port, and publish 3 macros: sNAME_addr, sNAME_port and sNAME_sock, but only once they are started. For varnishtest to create the vcl with the correct values, the server must be started when you use -vcl+backend.
It's a string, either double-quoted "like this", but most of the time enclosed in curly brackets, allowing multilining. Write a command per line in it, empty line are ignored, and long line can be wrapped by using a backslash. For example:
client c1 {
txreq -url /foo \
-hdr "bar: baz"
rxresp
} -run
Close the potential current connection, and accept a new one. Note that this new connection is H/1.
Close the connection. Not that if operating in H/2 mode, no extra (GOAWAY) frame is sent, it's simply a TCP close.
Close the connection. Not that if operating in H/2 mode, no extra (GOAWAY) frame is sent, it's simply a TCP close.
Test if "STRING1 OP STRING2" is true, and if not, fails the test. OP can be ==, <, <=, >, >= when STRING1 and STRING2 represent numbers in which case it's an order operator. If STRING1 and STRING2 are meant as strings OP is a matching operator, either == (exact match) or ~ (regex match).
varnishtet will first try to resolve STRING1 and STRING2 by looking if they have special meanings, in which case, the resolved value is use for the test. Note that this value can be a string representing a number, allowing for tests such as:
expect req.http.x-num > 2
Here's the list of recognized strings, most should be obvious as they either match VCL logic, or the txreq/txresp options:
Reads from the connection, expecting nothing to read but an EOF.
Reads from the connection, expecting nothing to read but an EOF.
Receive a preface, and if it matches, sets the server to H/2, aborts otherwise.
Send an H/2 preface ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") and set client to H/2.
Send a minimal request or response, but overload it if necessary.
txreq is client-specific and txresp is server-specific.
The only thing different between a request and a response, apart from who can send them is that the first line (request line vs status line), so all the options are prety much the same.
These three switches can appear in any order but must come before the following ones.
You can then use the arguments related to the body:
H/2 introduces the concept of streams, and these come with their own specification, and as it's quite big, have bee move to their own chapter.
This is very similar to the the shell command, except it takes a first string as argument before the command:
err_shell "foo" "echo foo"
err_shell expect the shell command to fail AND stdout to match the string, failing the test case otherwise.
Test that the required feature(s) for a test are available, and skip the test otherwise. feature takes any number of arguments from this list:
Reads the VSL and looks for records matching a given specification. It will process records trying to match the first pattern, and when done, will continue processing, trying to match the following pattern. If a pattern isn't matched, the test will fail.
logexpect threads are declared this way:
logexpect lNAME -v <id> [-g <grouping>] [-d 0|1] [-q query] \
[vsl arguments] {
expect <skip> <vxid> <tag> <regex>
expect <skip> <vxid> <tag> <regex>
...
} [-start|-wait]
And once declared, you can start them, or wait on them:
logexpect lNAME <-start|-wait>
With:
VSL arguments (similar to the varnishlog options):
And the arguments of the specifications lines are:
Pass the string given as argument to a shell. If you have multiple commands to run, you can use curly barces to describe a multi-lines script, eg:
shell {
echo begin
cat /etc/fstab
echo end
}
The vtc will fail if the return code of the shell is not 0.
(note: this section is at the top-level for easier navigation, but it's part of the client/server specification)
Streams map roughly to a request in H/2, a request is sent on stream N, the response too, then the stream is discarded. The main exception is the first stream, 0, that serves as coordinator.
Stream syntax follow the client/server one:
stream ID [SPEC] [ACTION]
ID is the H/2 stream number, while SPEC describes what will be done in that stream.
Note that, when parsing a stream action, if the entity isn't operating in H/2 mode, these spec is ran before:
txpri/rxpri # client/server
stream 0 {
txsettings
rxsettings
txsettings -ack
rxsettings
expect settings.ack == true
} -run
And H/2 mode is then activated before parsing the specification.
The specification of a stream follows the exact same rules as one for a client or a server.
These four commands are about sending headers. txreq, txresp will send HEADER frames, txcont will send CONTINUATION frames, and txpush PUSH frames. The only difference between txreq and txresp are the default headers set by each of them.
Insert an literal, indexed header. The first argument specify if the header should be added to the table, shouldn't, or mustn't be compressed if/when retransmitted.
INT is the idex of the header name to use.
The third argument informs about the Huffman encoding: yes (huf) or no (plain).
The last term is the literal value of the header.
Insert a literal header, with the same first argument as -litIdxHdr.
The second and third terms tell what the name of the header is and if it should be Huffman-encoded, while the last two do the same regarding the value.
By default, data frames are empty. The receiving end will know the whole body has been delivered thanks to the END_STREAM flag set in the last DATA frame, and txdata automatically set it.
These are two convenience functions to receive headers and body of an incoming request or response. The only difference is that rxreq can only be by a server, and rxresp by a client.
This works like rxhdrs, expecting a PUSH frame and then zero or more CONTINUATION frames.
rxhdrs will expect one HEADER frame, then, depending on the arguments, zero or more CONTINUATION frame.
Receiving data is done using the rxdata keywords and will retrieve one DATA frame, if you wish to receive more, you can use these two convenience arguments:
Push bytes directly on the wire. sendhex takes exactly one argument: a string describing the bytes, in hex notation, will possible whitespaces between them. Here's an example:
sendhex "00 00 08 00 0900 8d"
Receive a GOAWAY frame
Possible options include:
Receive a PING frame
Send PING frame.
Receive a PRIORITY frame
Send a PRIORITY frame
Receive a RST_STREAM frame
Send a RST_STREAM frame. By default, txrst will send a 0 error code (NO_ERROR).
Receive a SETTINGS frame
SETTINGS frames must be acknowledge, arguments are as follow (most of them are from rfc7540#6.5.2):
Receive a WINDOW_UPDATE frame
Transmit a WINDOW_UPDATE frame, increasing the amount of credit of the connection (from stream 0) or of the stream (any other stream).
expect in stream works as it does in client or server, except that the elements compared will be different.
Most of these elements will be frame specific, meaning that the last frame received on that stream must of the correct type.
Here the list of keywords you can look at.
Define and interact with varnish instances.
To define a Varnish server, you'll use this syntax:
varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING]
[-errvcl STRING STRING] [-jail STRING] [-proto PROXY]
The first varnish vNAME invocation will start the varnishd master process in the background, waiting for the -start switch to actually start the child.
With:
You can decide to start the Varnish instance and/or wait for several events:
varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped]
Once Varnish is started, you can talk to it (as you would through varnishadm) with these additional switches:
varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING]
[-expect STRING OP NUMBER]
Look into the VSM and make sure the counter identified by STRING has a correct value. OP can be ==, >, >=, <, <=. For example:
varnish v1 -expect SMA.s1.g_space > 1000000
This should be the first command in your vtc as it will identify the test case with a short yet descriptive sentence. It takes exactly one argument, a string, eg:
varnishtest "Check that varnishtest is actually a valid command"
It will also print that string in the log.
This document has been written by Guillaume Quintard.
This document is licensed under the same licence as Varnish itself. See LICENCE for details.