Bebop

Bebop
Login

Bebop

Bebop is a simple and fast serialisation format. You write bebop schemas, and from these bop files different tools produce source code in a number of different languages for serialising and deserialising data to and from the bebop wire format.

This repo builds a tool that generates Go source code from bop files.

One way you could use this is:

  1. go get wellquite.org/bebop/cmd/bebop@latest
  2. Create your bop file, e.g. protocol.bop
  3. Alongside it, create a generate.go file, with the following content:

    package mypackage
    
    //go:generate go run wellquite.org/bebop/cmd/bebop -i ./protocol.bop -o ./protocol.go -p my/full/package/name/mypackage
    
  4. Then, whenever you do a go generate ./... in your project, this bebop tool will generate protocol.go from your protocol.bop schema definition.

What features are supported?

Basically everything I could figure out from the upstream documentation; apart from the readonly flag which can't really be implemented in Go.

Dates do not interoperate

A date written by this library is not readable by other bebop implementations, and theirs are not readable here. Every other type is fine; a schema with no date field interoperates normally.

The wire format carries a date as a uint64 count of 100 nanosecond units. The upstream specification counts those from midnight, 1st January 0001 UTC — the origin .NET uses for DateTime.Ticks. This library counts them from the Unix epoch instead, which is 1969 years (621355968000000000 ticks) later. So:

a date of 2026-08-05 is read as
written by this library, read by a conforming implementation 0057-08-05
written by a conforming implementation, read by this library 3995-08-05

Nothing in the format distinguishes the two readings, so this goes wrong silently rather than failing.

I can't fix it: stored data already uses this encoding, and moving the origin would shift every date in it by 1969 years.

If you need to exchange dates anyway

Declare the field as a uint64 rather than a date, and convert with the two functions the runtime provides:

// to send
ticks, err := runtime.DateToSpecTicks(when)

// to receive
when := runtime.DateFromSpecTicks(ticks)

Going through a uint64 keeps the value clear of this library's date encoding entirely, which is what makes it portable. The pair is exact over the whole range the specification can hold — 1st January 0001 to 31st December 9999 UTC — which reaches back further than a date field here can express at all. DateToSpecTicks returns ErrDateOutsideSpecRange for anything outside that.

The specification additionally reserves the top two bits of the count for a .NET DateTime.Kind, which a conforming reader ignores and DateFromSpecTicks discards. A conforming writer never sets them, its latest date being in year 9999, but a date field here does for anything after year 16583.

Generated API

Imagine a bebop enum, struct, message or union, named Foo. This tool will generate the following API:

// EncodeBebop writes the value to the writer, serialized as Bebop.
func (*Foo) EncodeBebop(writer io.Writer) error

// DecodeBebop attempts to read Bebop from the reader and to
// deserialize it into the value, within runtime.DefaultLimits.
func (*Foo) DecodeBebop(reader io.Reader) error
func (*Foo) DecodeBebopLimited(reader io.Reader, limits runtime.Limits) error

// MarshalBebop writes the value into the buf, serialized as
// Bebop. The slice of the buf written to is returned. If the buf is too
// small, a new buf is created, written to, and returned.
func (*Foo) MarshalBebop(buf []byte) ([]byte, error)

// UnmarshalBebop attempts to read Bebop from the buf and to
// deserialize it into the value, within runtime.DefaultLimits.
func (*Foo) UnmarshalBebop(buf []byte) (int, error)
func (*Foo) UnmarshalBebopLimited(buf []byte, limits runtime.Limits) (int, error)

// SizeBebop returns the number of bytes this value uses when
// serialized to Bebop.
func (*Foo) SizeBebop() int

If you have specified an opcode, there will be:

// [opcode(653)]
func (*Foo) Opcode() uint32

Because I support imports, a few methods must be public that one would rather keep private: a generated type in one package has to be encoded and decoded partway through a buffer or stream belonging to a generated type in another package, and Go has no visibility level between "unexported" and "visible to everyone". Those methods are:

func (*Foo) BebopEncodeEncoder(encoder *runtime.Encoder) error
func (*Foo) BebopDecodeDecoder(decoder *runtime.Decoder) (int, error)
func (*Foo) BebopMarshalAt(buf []byte, offset int) (int, error)
func (*Foo) BebopUnmarshalAt(buf []byte, offset int, allow runtime.Allowance) (int, error)

They are exactly the runtime.Wire interface, which every generated type asserts it satisfies, and they all carry a // Not intended for public use comment. Don't call them: they work at a caller-supplied offset and do no framing of their own, so using them directly will produce malformed output. Use the five methods above instead.

Nothing else is exported for the benefit of generated code. In particular the array and map codecs are emitted afresh into each file that needs them, so they are unexported.

Limits on decoding

How much a value allocates, and how deeply it nests, are both named by the encoded value itself, and nothing about a buffer or a stream says whether those figures are honest. Four bytes can otherwise ask for tens of gigabytes, and Go reports that as a fatal error rather than a panic: the process dies, and no amount of care in the caller recovers it. A deep enough value exhausts the stack the same way.

DecodeBebop and UnmarshalBebop therefore decode within runtime.DefaultLimits(), which is 64MiB and a nesting depth of 100. The size is a budget spent across the whole value, not a cap on any one allocation, because an empty array costs four bytes on the wire and twenty four as a Go slice header, so a nest of arrays would otherwise multiply its input by the nesting depth.

To decode something larger, deeper, or from a sender you trust less:

func (*Foo) UnmarshalBebopLimited(buf []byte, limits runtime.Limits) (int, error)
func (*Foo) DecodeBebopLimited(reader io.Reader, limits runtime.Limits) error

Leave a field at zero to take the default for it, and raise a limit by setting a larger one, so runtime.Limits{MaxBytes: 1 << 30} allows a gigabyte and leaves the depth at 100. There is no value meaning no limit: there is always some size past which a value isn't worth decoding, and naming it costs nothing.

Encoding is unaffected: nothing there comes off the wire.