Free stage 2 of 3: runs on your computer.
This is one piece of the broker that keeps a greenhouse of a thousand sensors talking to its grower.
Stage 2: Read packets from a TCP stream
TCP delivers a stream of bytes, not a list of messages. One packet can arrive in several pieces, and several packets can arrive glued together. Your broker must cut the stream into packets itself.
Why it matters
Sensors in the cold corner of the greenhouse sit behind a weak Wi-Fi link, so their readings arrive in pieces, or glued to the next one. A broker that assumes one read equals one packet quietly drops those readings, and the grower never knows the cold zone went silent. Cutting the stream correctly keeps every reading.
Your task
Read each packet as: first byte, remaining length, then exactly that many bytes.
The remaining length uses a variable-length encoding: 7 data bits per byte, and the high bit says "another length byte follows". It is at most 4 bytes long.
| value | encoded |
|--------------|---------------|
| 0 | 00 |
| 127 | 7F |
| 128 | 80 01 |
| 16383 | FF 7F |
| 16384 | 80 80 01 |
| 268435455 | FF FF FF 7F |
Behaviors the tests check:
- A
CONNECTis accepted whatever its size, and a packet sent right behind it in the same TCP segment is not lost. Once connected, aPINGREQ(C0 00) is answered with aPINGRESP(D0 00). - Bytes that arrive one at a time, or split in the middle of the remaining length, are reassembled. Your broker must not answer before the packet is complete.
- A remaining length that is still not finished after 4 bytes is invalid: close the connection.
- If the client hangs up in the middle of a packet, close your side and keep serving other clients. Announcing a huge length (up to 268 435 455) and sending nothing must not make you allocate that much memory.
Run the tests
iotbricks test local --stage 2 --run ./iotbricks-mqtt-go
Hints
Hint 1: the classic bug
A single conn.Read may return half a packet, or one and a half. Use a read that blocks until it
has exactly the number of bytes you ask for (io.ReadFull in Go, a read loop elsewhere), and wrap
the connection in a buffered reader so reading byte by byte stays cheap.
Hint 2: decoding the remaining length by hand
80 01 means: first byte 0x80 has the high bit set, so its data is 0x00 and another byte
follows. The second byte 0x01 has no high bit, so it is the last. Each byte weighs 128 times more
than the previous one: 0 * 1 + 1 * 128 = 128.
Hint 3: the algorithm from the spec
multiplier = 1
value = 0
repeat:
byte = read one byte
value += (byte & 0x7F) * multiplier
if multiplier > 128*128*128: malformed, close
multiplier *= 128
until (byte & 0x80) == 0
Then validate value against a sane maximum before allocating a buffer of that size.
1Run the tests on your broker
iotbricks test local --stage 2 --run ./iotbricks-mqtt-go
--run is the command that starts your broker, for example --run "go run .". The broker must listen on the port in the MQTT_PORT environment variable (default 1883).