Usage with xpra start-desktop

[IncognitoTGT]

I'm implementing this client with one of my apps, but have had trouble connecting to it with an xpra server in start-desktop mode. When I try to use the integration steps, it shows that it's connected (according to the console), but trying to obtain the canvas with methods such as checking window 0, have resulted in failure.

[andersevenrud]

Hi,

I haven't looked at this project in a really long time, so I need some details 😅

Could you provide exact steps to reproduce and some screenshots/dumps ?

[IncognitoTGT]

create an instance with xpra start-desktop, and then try to connect with it. I used this react component:

"use client";
import { useEffect, useImperativeHandle, useRef, useState } from "react";
import { XpraClient, XpraWindowManager } from "xpra-html5-client";
import type { XpraClientEventEmitters, XpraConnectionOptions } from "xpra-html5-client";
export interface XpraHandle {
	wm: XpraWindowManager | null;
	client: XpraClient | null;
}
export interface Props {
	ref?: React.Ref<XpraHandle>;
	events?: Partial<XpraClientEventEmitters>;
	connectionOptions: [string, Partial<XpraConnectionOptions>];
	loader?: React.ReactNode;
	className?: string;
}
export default function XpraScreen({ events, connectionOptions, ref, loader, className }: Props) {
	const xpraRef = useRef<XpraClient | null>(null);
	const windowManagerRef = useRef<XpraWindowManager | null>(null);
	const divRef = useRef<HTMLDivElement | null>(null);
	const [inited, setInited] = useState(false);
	const [size, setSize] = useState<[number, number]>([window.innerWidth, window.innerHeight]);
	useEffect(() => {
		const handler = () => setSize([window.innerWidth, window.innerHeight]);
		window.addEventListener("resize", handler);
		return () => window.removeEventListener("resize", handler);
	}, []);
	useEffect(() => {
		try {
			const worker = new Worker(new URL("./workers/main.ts", import.meta.url));
			const decoder = new Worker(new URL("./workers/decode.ts", import.meta.url));
			xpraRef.current = new XpraClient({ worker, decoder });
			(async () => {
				await xpraRef.current?.init();
				setInited(true);
			})();
			if (inited) {
				windowManagerRef.current = new XpraWindowManager(xpraRef.current);
				windowManagerRef.current?.init();
				windowManagerRef.current.createWindow({
					id: 0,
					position: [0, 0],
					dimension: size,
					metadata: {
						"window-type": ["NORMAL"],
						"class-instance": [],
						title: "Desktop",
					},
					clientProperties: {},
					overrideRedirect: false,
				});
				const { canvas } = windowManagerRef.current.getWindow(0) || {};
				if (canvas) {
					canvas.style.zIndex = "30";
					divRef.current?.appendChild(canvas);
				}
				if (events) {
					for (const [event, fn] of Object.entries(events)) {
						xpraRef.current.on(event as keyof XpraClientEventEmitters, fn);
					}
				}
				xpraRef.current.connect(...connectionOptions);
			}
		} catch (e) {
			xpraRef.current?.emit("error", e as string);
		}
		return () => {
			if (!inited) return;
			if (events) {
				for (const [event, fn] of Object.entries(events)) {
					xpraRef.current?.off(event as keyof XpraClientEventEmitters, fn);
				}
			}
			xpraRef.current?.disconnect();
			xpraRef.current = null;
		};
	}, [events, connectionOptions, inited, size]);
	useImperativeHandle(ref, () => ({
		wm: windowManagerRef.current,
		client: xpraRef.current,
	}));
	return (
		<>
			{!inited ? loader || "Loading..." : null}
			<div
				className={className}
				ref={divRef}
				onContextMenu={(e) => e.preventDefault()}
				onMouseUp={(e) => windowManagerRef.current?.mouseButton(null, e.nativeEvent, false)}
				onMouseDown={(e) => windowManagerRef.current?.mouseButton(null, e.nativeEvent, true)}
				onMouseMove={(e) => windowManagerRef.current?.mouseMove(null, e.nativeEvent)}
			/>
		</>
	);
}

console logs look normal

[andersevenrud]

Does the same issue occur if you try doing this without a React component ?

[IncognitoTGT]

It loads on your client properly.

[andersevenrud]

I am not at home, so I cannot look into your code, but I suggest looking at the client app provided in this repo.

It uses a context to provide an Xpra Client instance so that it has it's own lifecycle outside the component(s) that you might be able to re-use. Using effects etc. inside a component can get tricky when dealing with instancing.

[IncognitoTGT]

Instancing isn't the problem here, it's actually trying to get the canvas to actually display stuff. It gets rendered, but nothing's in it