TRF Core Functionality

Telescope Remote Framebuffer Library.

Defines

TRF_RECV_CQ
TRF_SEND_CQ
TRF_FI_MIN_VER
TRF_FI_MAX_VER
TRF_FI_CUR_VER

Functions

static inline int trfGetDeadline(struct timespec *out, int delay_ms)

Get a timespec corresponding to a deadline in the future.

Parameters:

  • out – Pointer to timespec to be filled in

  • delay_ms – Delay in milliseconds

Returns:

0 on success, negative error code on failure.

int trfCreateSubchannel(PTRFContext ctx, PTRFContext *ctx_out, uint32_t id)

Create a subchannel.

Parameters:

  • ctx – Allocated context containing the main channel.

  • ctx_out – Subchannel output.

  • id – Subchannel ID. Must be unique to the main context.

Returns:

0 on success, negative error code on failure.

int trfProcessSubchannelReq(PTRFContext ctx, PTRFContext *ctx_out, TrfMsg__MessageWrapper *req)

Process a subchannel creation request from the peer.

Parameters:

  • ctx – Allocated context containing the main channel.

  • ctx_out – Subchannel output.

  • req – TRF control message containing the subchannel creation request.

Returns:

0 on success, negative error code on failure.

int trfBindSubchannel(PTRFContext main, PTRFContext sub)

Bind the subchannel to the main context. This ensures that the subchannel is closed when the main context is destroyed.

Parameters:

  • main – Main channel.

  • sub – Subchannel.

Returns:

0 on success, negative error code on failure.

int trfUnbindSubchannel(PTRFContext main, uint32_t id)

Unbind the subchannel from the main context. This is required if the channel disconnects, to prevent invalid memory accesses.

Parameters:

  • main – Main channel

  • id – Subchannel ID

Returns:

0 on success, negative error code on failure.

int trfAllocActiveEP(PTRFXFabric ctx, struct fi_info *fi, void *data, size_t size)

Allocate an active endpoint.

See also

trfGetName()

Parameters:

  • ctx – Context in which to store the allocated endpoint.

  • fi – Endpoint parameters, returned by fi_getinfo()

  • data – If a source address was not assigned in the fi_info struct, this parameter may be used to specify the source address. The address format of this parameter must match fi->addr_format. To determine the address that was assigned,

  • size – Size of the data buffer

Returns:

0 on success, -errno on failure.

int trfInsertAVSerialized(PTRFXFabric ctx, char *addr, fi_addr_t *out)

Insert a serialized address in FI_ADDR_STR or TRF format into the context’s address vector.

Parameters:

  • ctx – Context in which to store the address.

  • addr – Serialized address to insert.

  • out – If the address is successfully inserted, this parameter will be set to an address identifier.

Returns:

0 on success, negative error code on failure.

int trfDeserializeWireProto(const char *proto, uint32_t *out)

Deserialize a TRF wire protocol string into a format understood by the fabric transport.

Parameters:

  • proto – Serialized protocol string

  • out – Output fabric protocol indentifier

Returns:

0 on success, negative error code on failure.

int trfSerializeWireProto(uint32_t proto, char **out)

Serialize a fabric transport protocol name into a string for transmission over the network.

Parameters:

  • proto – Protocol identifier.

  • out – Output string.

Returns:

0 on success, negative error code on failure.

PTRFContext trfAllocContext()

Allocate a TRFContext.

This does not allocate any resources within the context; you must use other libtrf functions such as trfNCClientInit() to allocate resources.

Returns:

PTRFContext

int trfGetFabricProviders(const char *host, const char *port, enum TRFEPType req_type, struct fi_info **fi_out)

Get the list of available fabric providers which can reach the target.

See also

enum TRFEPType

Parameters:

  • host – Hostname of the target

  • port – Port or service name of the target

  • req_type – Endpoint type

  • fi_out – Output list of available fabric providers

Returns:

0 on success, negative error code on failure.

void trfFreeInterfaceList(PTRFInterface ifaces)

Free an interface list.

This call frees a linked list of struct TRFInterface, typically allocated using the trfGetInterfaceList() call.

Parameters:

ifaces – Interface list to free.

int trfCreateChannel(PTRFContext ctx, struct fi_info *fi, void *data, size_t size)

Allocate RD communication channel.

This call creates one side of a FI_EP_RDM channel from Libfabric according to the supplied arguments.

See also

struct TRFXFabric

Parameters:

  • ctx – Allocated context to store the channel. The context is stored within ctx->xfer.fabric.

  • fi – Fabric provider to use, returned from a fi_getinfo() or trfGetFabricProviders() call.

  • data – If a source address was not assigned in the fi_info struct, this parameter may be used to specify the source address. The address format of this parameter must match fi->addr_format.

  • size – Size of the data buffer.

Returns:

0 on success, negative error code on failure.

int trfGetRoute(const char *dst, const char *prov, const char *proto, struct fi_info **fi)

Determine fabrics which might reach the specified route.

Note: This function is designed for dynamic port allocation, and as such does not assign a source address to the returned fi_info list.

Parameters:

  • dst – Destination address to use, in TRF Serialized format.

  • prov – Provider name, e.g. “verbs;ofi_rxm”

  • proto – Wire protocol e.g. FI_PROTO_RDMA_CM_IB_RC

  • fi – Output list of available fabrics.

Returns:

0 on success, negative error code on failure.

int trfConvertFabricAF(uint32_t fi_addr_format)

Convert internal Libfabric address format identifier into a TRF format identifier to be sent over the network.

Parameters:

fi_addr_format – Libfabric address format identifier e.g. FI_ADDR_STR

Returns:

TRF address format identifier e.g. TRFX_SOCKADDR

int trfConvertInternalAF(uint32_t trf_addr_format)

Convert TRF address format identifier into Libfabric address format.

Parameters:

trf_addr_format – TRF address format identifier e.g. TRFX_SOCKADDR

Returns:

Libfabric address format identifier e.g. FI_ADDR_STR

int trfSerializeAddress(void *data, enum TRFXAddr format, char **out)

Serialize an address from an internal format, such as struct sockaddr, into the TRF network transmission format.

Parameters:

  • data – Address to serialize, in the raw format

  • format – The format of the data referenced by the data pointer

  • out – Output buffer pointer to be allocated and set to the serialized data, e.g. trfx_sockaddr://1.2.3.4:5678

int trfDeserializeAddress(const char *ser_addr, int data_len, void **data, int *format)

Deserialize a TRF address into an internal format, such as struct sockaddr.

Parameters:

  • ser_addr – Serialized address

  • data_len – Length of the serialized address

  • data – Output buffer pointer to be allocated and set to the deserialized data, e.g. struct sockaddr

  • format – Detected format of the output data

Returns:

int

int trfRegBuf(PTRFXFabric ctx, void *addr, size_t len, uint64_t flags, struct fid_mr **mr_out)

Register a memory buffer.

Parameters:

  • ctx – Allocated context to store the memory region. The context must have been initialized, with a fabric and domain.

  • addr – Address of the memory buffer to register.

  • len – Length of the memory buffer.

  • flags – Protection flags for the memory region.

  • mr_out – Output pointer to the memory region.

Returns:

0 on success, negative error code on failure.

int trfRegInternalMsgBuf(PTRFContext ctx, void *addr, size_t len)

Register a memory buffer for sending and receiving messages.

Parameters:

  • ctx – Allocated context to store the memory region. The context must have been initialized, with a fabric and domain.

  • addr – Address of the memory buffer to register.

  • len – Length of the memory buffer.

Returns:

0 on success, negative error code on failure.

int trfGetEndpointName(PTRFContext ctx, char **sas_buf)

Get Libfabric endpoint name.

Parameters:

  • ctx – Context containing initialized fabric context.

  • sas_buf – Pointer to a buffer to be allocated and set.

Returns:

0 on success, negative error code on failure

void trfDestroyFabricContext(PTRFContext ctx)

Destroys and frees all fabric context items stored within a TRF context object.

Parameters:

ctx – Context containing an initialized fabric context.

void trfSendDisconnectMsg(int fd, uint64_t session_id)

Sends a disconnect Message.

Parameters:

  • fd – File descriptor to send disconnect message on

  • session_id – Session_ID

Returns:

nothing is returned

void trfDestroyContext(PTRFContext ctx)

Destroy PTRFContext and free memory.

Parameters:

ctx – PTRFContext to destroy

Returns:

nothing is retured

void *trfAllocAligned(size_t size, size_t alignment)

Allocate an aligned block of memory.

Parameters:

  • size – Size of the memory block to allocate.

  • alignment – Alignment of the memory block.

Returns:

Pointer to the allocated memory block, or NULL on failure.

int trfBindDisplayList(PTRFContext ctx, PTRFDisplay list)

Bind list of displays to context.

Parameters:

  • ctx – Context to bind displays to

  • list – List of displays to bind

Returns:

return 0 if successful, negative error code on failure

int trfUpdateDisplayAddr(PTRFContext ctx, PTRFDisplay disp, void *addr)

Update the memory address of the referenced display’s framebuffer, performing memory deregistration and re-registration.

Note: This only disables RMA access to the old framebuffer, and does not free the old framebuffer’s memory. It is the responsibility of the caller to do so after calling this function.

Parameters:

  • ctx – Context the display is bound to

  • disp – Display to update

  • addr – New address

Returns:

0 on success, negative error code on failure

static inline ssize_t trfGetTextureBytes(size_t width, size_t height, enum TRFTexFormat fmt)

Determine the number of bytes required to store a given texture.

Parameters:

  • width – Width

  • height – Height

  • fmt – Texture format.

Returns:

Number of bytes required to store the texture. Negative error code on failure.

static inline ssize_t trfGetDisplayBytes(PTRFDisplay disp)

Get number of bytes needed to store the contents of the display.

Parameters:

disp – Display

Returns:

ssize_t

static inline size_t trfGetCursorBytes(PTRFCursor cur)
static inline uint8_t trfTextureIsCompressed(enum TRFTexFormat fmt)
void trfFreeDisplayList(PTRFDisplay disp, int dealloc)

Free a display list.

Parameters:

  • disp – Display list to free

  • dealloc – Whether to deallocate and deregister memory occupied by framebuffer data. If this display list contains shared buffer resources, it is advised to set this to 0.

int trfGetMessageAuto(PTRFContext ctx, uint64_t flags, uint64_t *processed, void **data_out, int *opaque)

Automatically process messages internally based on the message type.

Parameters:

  • ctx – Context to use

  • flags – Messages to process internally

  • processed – Message that was processed

  • data_out – Message data. Typically this is a pointer to a TrfMsg__MessageWrapper, but can be any data. This is set to a non-NULL value if the message wasn’t processed internally.

  • opaque – Opaque pointer, used internally for state tracking. Do not modify. Before the first call, the initial value of this variable should be set to 0.

Returns:

Number of messages processed internally, negative error code if an error occurred.

int trfRegDisplaySource(PTRFContext ctx, PTRFDisplay disp)

Make the referenced display buffer available for use, as the source (server) buffer.

Parameters:

  • ctx – Initialized context to make the display buffer available to.

  • disp – Display to make available. Note: if this contains multiple displays, only the first display will be made available.

Returns:

0 on success, negative error code on failure.

int trfRegDisplaySink(PTRFContext ctx, PTRFDisplay disp)

Make the referenced display buffer available for use, as the sink (client) buffer.

Parameters:

  • ctx – Initialized context to make the display buffer available to.

  • disp – Display to make available. Note: if this contains multiple displays, only the first display will be made available.

Returns:

0 on success, negative error code on failure.

int trfRegDisplayCustom(PTRFContext ctx, PTRFDisplay disp, size_t size, size_t offset, uint64_t flags)

Custom registration routine. This is used when the framebuffer memory region contains other data.

Parameters:

  • ctx – Initialized context to make the display buffer available to.

  • disp – Display to make available.

  • size – Size of the display buffer.

  • offset – Offset between the start of the MR and the start of the frame data.

PTRFDisplay trfGetDisplayByID(PTRFDisplay disp_list, int id)

Get display by ID.

Parameters:

  • disp_list – Display list

  • id – ID

Returns:

Pointer to item in linked list with specified ID. If NULL, the operation failed and errno will be set to indicate the reason.

int trfAckClientReq(PTRFContext ctx, uint32_t *disp_ids, int n_disp_ids)

Acknowledge the client’s display initialization request.

Parameters:

  • ctx – Context to use

  • disp_ids – Initialized display ID array

  • n_disp_ids – Number of display IDs in the array

Returns:

0 on success, negative error code on failure

static inline void *trfGetFBPtr(PTRFDisplay disp)

Get the actual framebuffer data pointer after offsets.

Parameters:

disp – Display to use.

Returns:

void *

static inline ssize_t trfSendFrame(PTRFContext ctx, PTRFDisplay disp, uint64_t rbuf, uint64_t rkey)

Send a frame to the client.

Parameters:

  • ctx – Initialized context to send the frame to.

  • disp – Display identifier.

  • rbuf – Pointer to the remote frame buffer.

  • rkey – Remote access key.

Returns:

0 on success, negative error code on failure.

ssize_t trfSendFramePart(PTRFContext ctx, PTRFDisplay disp, uint64_t rbuf, uint64_t rkey, struct TRFRect *rects, size_t num_rects)

Send a partial frame update.

Performs a fabric RMA write operation to the destination buffer. The client is not informed of the operation status; once the operation completes, the server should send an acknowledgement.

Compressed textures are not supported.

Parameters:

  • ctx – Initialized context to send the frame to.

  • disp – Display identifier.

  • rbuf – Pointer to the remote frame buffer.

  • rkey – Remote access key.

  • rects – Rectangles corresponding to regions of the framebuffer to be updated.

  • num_rects – Number of rectangles in the list.

Returns:

0 on success, negative error code on failure.

ssize_t trfSendFrameChunk(PTRFContext ctx, PTRFDisplay disp, size_t start, size_t end, uint64_t rbuf, uint64_t rkey)

Send a partial frame update, based on byte offsets.

Performs a fabric RMA write operation to the destination buffer. The client is not informed of the operation status; once the operation completes, the server should send an acknowledgement.

Compressed textures are not supported.

Parameters:

  • ctx – Initialized context to send the frame to.

  • disp – Display identifier.

  • start – Start offset in the framebuffer.

  • end – End offset in the framebuffer.

  • rbuf – Pointer to the remote frame buffer.

  • rkey – Remote access key.

Returns:

0 on success, negative error code on failure.

ssize_t trfRecvFrame(PTRFContext ctx, PTRFDisplay disp)

Send a frame receive request. Non-blocking operation.

Parameters:

  • ctx – Initialized context.

  • disp – Display identifier, containing registered framebuffer region.

Returns:

0 on success, negative error code on failure.

int trfGetServerDisplays(PTRFContext ctx, PTRFDisplay *out)

Get the list of displays available to the client from the server.

Parameters:

  • ctx – Context

  • out – Server display list to be allocated and filled.

Returns:

0 on success, negative error code on failure.

int trfSendDisplayList(PTRFContext ctx)

Send a display list to the client contained within the current context.

The client should have requested display metadata from the server.

Parameters:

ctx – Initialized context containing display list.

Returns:

0 on success, negative error code on failure.

int trfSendClientReq(PTRFContext ctx, PTRFDisplay disp)

Send a display initialization request to the server.

Parameters:

  • ctx – Initialized context.

  • disp – Display to initialize.

Returns:

0 on success, negative error code on failure.

int trfAckFrameReq(PTRFContext ctx, PTRFDisplay display)

Send an confirmation to the client that a frame has been updated.

Parameters:

  • ctx – Context to use.

  • display – Display the acknowledgement is for.

Returns:

0 on success, negative error code on failure.

int trfSendKeepAlive(PTRFContext ctx)

Send Keep alive message.

Parameters:

ctx – Context to use

Returns:

0 on success, negative error code

ssize_t trfGetSendProgress(PTRFContext ctx, struct fi_cq_data_entry *de, struct fi_cq_err_entry *err, size_t count, PTRFContextOpts opts)

Get the progress of all send operations.

Parameters:

  • ctx – Context to use.

  • de – Data entry containing successful completion details.

  • err – Data entry containing error details, if any

  • count – Maximum number of completions to receive.

  • opts – Optional options to override the default context behaviours.

Returns:

Number of completed operations. Negative error code on failure.

ssize_t trfGetRecvProgress(PTRFContext ctx, struct fi_cq_data_entry *de, struct fi_cq_err_entry *err, size_t count, PTRFContextOpts opts)

Get the progress of all receive operations.

Parameters:

  • ctx – Context to use.

  • de – Data entry containing successful completion details.

  • err – Data entry containing error details, if any

  • count – Maximum number of completions to receive.

  • opts – Optional options to override the default context behaviours.

Returns:

Number of completed operations. Negative error code on failure.