Project BASE - Faster HTTP Reconnect

[evnchn]

This PR introduces a HTTP-based special mechanism for loading in the page which is must faster than the browser.

Brief explainer

The server, upon the setting of a special private header 'X-NiceGUI-Client-ID', instead of serving HTML, proceeds instead to serve it in JSON machine-readable format.

Which, the special JavaScript function softReload will:

• Disconnect the old Socket.IO
• Fetch with special header and parse JSON
• Re-run createApp(...)
• Dynamically import any JS modules and add them to add component
• Re-execute vue_script for the Plotly and similar elements which rely on it
• Mount the app to #app

Essentially, it does what a page load will do, but without a page load.

---

Name meaning of BASE

BASE stands for "Because Asked Server Explicitly", which is an (admittedly not very good) acryonym reflecting on the nature that the client asks the server explicitly for reconnection advice via HTTP, including the crucial Client ID.

This contrasts with project ACID #4552 before it, which the client can say it had any arbitrary Client ID via Socket.IO, and the server must bend around it by then suddenly spawning in the Client, bypassing HTTP layer entirely.

---

Advantages

Advantages shared common between ACID and BASE:

• Fast page switching (implemented via softReload)
• Zero Flash Of Unstyled Content as a baseline guarantee.
• Fast reconnection in case client destroyed, because disconnected over the timeout duration / server reboot (to be implemented)

Advantages exclusive to BASE:

• Maintains the one-to-one relationship between HTTP request and NiceGUI Client instances
• Ensures Middlewares stay effective
• Avoids fallacies such as HTTP GET parameters being lost
• Practically zero-impact on userspace code, since server-side all new logic is basically not involved if 'X-NiceGUI-Client-ID' is not set in header
• May make creating custom NiceGUI frontends (like an ESP32 touchscreen UI) slightly easier, though not by much.

---

Remaining tasks

• Assess benefit VS impact
• Finish off the fast reconnection in client destroyed / server reboot situation
• softReload doesn't set browser URL bar
• Extensive testing (non-negotiable)
• Extensive documentation
• Drop unused fields from the JSONResponse if they are not needed by the reload process
• Can we make the typical page load in code and the softReload code to be unified? This can further ensure exact same behaviour.

---

Testing code

My testing code

import plotly.graph_objects as go
from nicegui import ui
from random import random


def menu():
    with ui.row():
        ui.label('Menu').classes('text-2xl text-gray-700')
        ui.button('Page 1', on_click=lambda: ui.navigate.to('/page1'))
        ui.button('Page 2', on_click=lambda: ui.navigate.to('/page2'))
        ui.button('Page 3', on_click=lambda: ui.navigate.to('/page3'))
        ui.button('Page 4', on_click=lambda: ui.navigate.to('/page4'))
        ui.button('Page 5', on_click=lambda: ui.navigate.to('/page5'))
    with ui.row():
        ui.button('Soft Page 1', on_click=lambda: ui.run_javascript('softReload("/page1")'))
        ui.button('Soft Page 2', on_click=lambda: ui.run_javascript('softReload("/page2")'))
        ui.button('Soft Page 3', on_click=lambda: ui.run_javascript('softReload("/page3")'))
        ui.button('Soft Page 4', on_click=lambda: ui.run_javascript('softReload("/page4")'))
        ui.button('Soft Page 5', on_click=lambda: ui.run_javascript('softReload("/page5")'))


@ui.page('/page1')
def page1():
    menu()
    ui.label('Page 1').classes('text-2xl text-red-500')


@ui.page('/page2')
def page2():
    menu()
    ui.label('Page 2').classes('text-2xl text-blue-500')


@ui.page('/page3')
def page3():
    menu()
    with ui.card():
        ui.mermaid('''
            graph TD
            A --> B
            A --> C
            A --> D
        ''')

# page4 with plotly


@ui.page('/page4')
def page4():
    menu()
    fig = go.Figure()
    fig.update_layout(margin=dict(l=0, r=0, t=0, b=0))
    plot = ui.plotly(fig).classes('w-full h-40')

    def add_trace():
        fig.add_trace(go.Scatter(x=[1, 2, 3], y=[random(), random(), random()]))
        plot.update()

    ui.button('Add trace', on_click=add_trace)

# page5 with dark mode


@ui.page('/page5')
def page5():
    menu()
    dark = ui.dark_mode()
    ui.switch('Dark mode').bind_value(dark)


menu()

ui.run()

Results:

• Switching through all 5 pages work seamlessly, despite starting on whichever page.
• Since Mermaid loaded in while starting from page 1, loading ES modules works.
• Since Plotly loaded in while starting from page 1, loading vue_scripts works.

[falkoschindler]

Very interesting idea, @evnchn! Seems to be a comparatively small change with a big impact. 👌🏻

[evnchn]

#4719 (comment) addressed with f955e42

[evnchn]

So @rodja and I was discussing how this PR relates to Single-Page Application in another thread.

As a recap, the definition of a Single-Page Application is:

1. Only one browser load
2. Capture browser's pushstate/popstate
3. Load in only the updated content

For this PR it is itself mostly oriented towards solving point 1, since as you know the single-megabit uplink hate page loads. However, as we have seen in main.py (the NiceGUI documentation), Project BASE is handling point 2 with just a bit of extra JavaScript. Check below:

nicegui/main.py

Lines 64 to 91 in fa34e46

	ui.add_body_html('''<script>
	// Your custom function
	function customFunction(event) {
	let target = event.target;
	// Traverse up the DOM tree to find the nearest anchor tag
	while (target && target.tagName !== 'A') {
	target = target.parentElement;
	}
	if (target && target.tagName === 'A') {
	console.log('Link clicked!'); // Replace with your custom logic
	console.log(target.href); // Log the href of the clicked link
	try {
	window.softReload(target.href);
	event.preventDefault();
	} catch (e) {
	console.error('Error in softReload:', e);
	}
	}
	}
	// Attach the event listener to the document
	document.addEventListener('click', customFunction);
	window.addEventListener("popstate", (event) => {
	console.log("popstate", event.state);
	softReload(window.location.href, event.state?.x, event.state?.y);
	});
	</script>
	''', shared=True)

Point 3 is real, though, and I should have thought of that as the "performance-focused guy" on the team 😅. Anyways, I think it is doable, if we:

1. Have the user cache all elements repeatedly sent via Browser Data Store: Retransmission Avoidance (-40% for docs page alone) #4796, or
2. Similar to the cookie approach in Browser Data Store: Retransmission Avoidance (-40% for docs page alone) #4796, we set the cookie indicating what Client is associated with ourselves right now. Such that, when we spawn the new Client, we can compare to the old Client and deduplicate the elements if it's clearly the same between the pages.

Point 2 elaboration: Proposed elements JSON string:

{"0":"SAME", "1":"SAME", ... "39": {"tag":"nicegui-link","text":"New Link...",... }

[rodja]

I'm not sure if a combination of this PR with a UI element cache can provide a sufficient SPA implementation. From my point of view, a NiceGUI SPA is not only defined by "Load in only the updated content" but also "Not rerun backend code". And to generate a JSON string like {"0":"SAME", "1":"SAME", ... "39": {"tag":"nicegui-link","text":"New Link...",... } you would need to re-execute the whole page function including headers etc for every sub-page-change.

[evnchn]

From my point of view, a NiceGUI SPA is not only defined by "Load in only the updated content" but also "Not rerun backend code"

I'd like to disagree by pointing out a half-baked (IMO) SPA, namely GitHub itself.

Ever noticed, when the number of PRs has gone up, that some old tabs stays on the old PR count, until you reload or go inside and outside of NiceGUI repository? I sure did, as the source of those PRs 😅

This is because, GitHub does not re-execute the backend code responsible to calculating how many PRs are there and update the header, while just pulling in the body.

But this could work for this PR:

def get_number_of_PRs():
    # some function

def common_header():
    ui.label(f'Number of PRs: {get_number_of_PRs()}')

@ui.page('/page1')
def page1():
    common_header()
    # content

@ui.page('/page2')
def page2():
    common_header()
    # content

Every single time you re-navigate using soft reload, it re-executes the common_header, and instructing the client to update with the new header if changed, or use the old one if not changed.

---

In contrast, assuming #4821 syntax, this would keep the stale PR count on navigation:

def get_number_of_PRs():
    # some function

def common_header():
    ui.label(f'Number of PRs: {get_number_of_PRs()}')

@ui.page('/')
@ui.page('/{_:path}') # tells FastAPI/Starlette that all path's should land on our SPA
def index():
    common_header()
    ui.sub_pages({'/': child, '/other': child2})

def child():
    ui.link('goto other', '/other')

def child2():
    ui.link('goto main', '/')

Rather, to change the common header, we need to actively pass the elements which may change into the subpage, as prescribed in #4821 (comment)

@ui.page('/')
@ui.page('/{_:path}')
def index():
    title = ui.label()
    ui.sub_pages({
        '/': lambda: child(title),
        '/other': lambda: child2(title)
    })

def child(title: ui.label):
    title.set_text('child title')
    ui.link('goto other', '/other')

def child2(title: ui.label):
    title.set_text('other title')
    ui.link('goto main', '/')

---

Overall, I am inclined to believe, if implemented well, this PR can be a better implementation to SPA than #4821.

[evnchn]

@rodja check this out 18f83e7

from nicegui import ui


def reused_ui_elements():
    ui.label('This is a label that can be reused.')
    ui.table(rows=[{'id': 1, 'name': 'Alice'}, {'id': 2, 'name': 'Bob'}])
    ui.markdown('#### This is a markdown element that can be reused.')
    ui.link('Go to main page', '/')
    ui.link('Go to another page', '/another')


@ui.page('/')
def main_page():
    ui.label('This is the main page.')
    reused_ui_elements()


@ui.page('/another')
def another_page():
    ui.label('This is another page.')
    reused_ui_elements()

# Copy from main.py
ui.add_body_html('''<script>
    // Your custom function
    function customFunction(event) {
        let target = event.target;
        // Traverse up the DOM tree to find the nearest anchor tag
        while (target && target.tagName !== 'A') {
            target = target.parentElement;
        }
        if (target && target.tagName === 'A') {
            console.log('Link clicked!'); // Replace with your custom logic
            console.log(target.href); // Log the href of the clicked link
            try {
                window.softReload(target.href);
                event.preventDefault();
            } catch (e) {
                console.error('Error in softReload:', e);
            }
        }
    }
    // Attach the event listener to the document
    document.addEventListener('click', customFunction);

    window.addEventListener("popstate", (event) => {
        console.log("popstate", event.state);
        softReload(window.location.href, event.state?.x, event.state?.y);
    });
    </script>
''', shared=True)


ui.run()

Betwen page soft reloads, it detects that the elements in reused_ui_elements() did not change.

As such, Project REBASE (name loosely inspired from git rebase) short-formed it to {"0":{"=":true}, ...}, reducing data transmission. If there were any changes, it would send them over.

Note that the table is re-transmitted, because I did not fully steal the logic from #4796. If I did, then it would detect that the table, only the event handlers are changed, and pass something like the below (transmitting only the new event handlers):

'{"6":{"=": true, "events":[{"listener_id":"d8134f52-fa89-4232-a9be-5f34f5636162","type":"selection","specials":[],"modifiers":[],"keys":[],"args":[["added","rows","keys"]],"throttle":0,"leading_events":true,"trailing_events":true,"js_handler":null},{"listener_id":"89168e4d-90d2-4d29-842c-af6ae29ffa9d","type":"update:pagination","specials":[],"modifiers":[],"keys":[],"args":null,"throttle":0,"leading_events":true,"trailing_events":true,"js_handler":null}]}}'

Seems like this is a much easier approach over SPA, with NiceGUI taking up the heavy lifting of finding identical elements between pages, instead of user having to divide their content between the main page and ui.sub_page.

[evnchn]

The logic in 21c2fad is:

• Server-side, mark via '=': True, such that replaceUndefinedAttributes base off its response from mounted_app.elements[id]
• Server-side, for keys in the current element, we add on top, if it is different to the past element
• Client-side, replaceUndefinedAttributes will take the past element, add on top the changed keys from the previous step
• Client-side, we update the page, while sending minimal data across.

[rodja]

I'm really fond of the friendly competition here, @evnchn. It is awesome how you transformed this "Faster HTTP Reconnect" into a SPA concept 😃

For me it is not an either #4821 or this PR. They server both quite different purposes. Even if this PR might be a the groundwork for an "SPA behind-the-scenes", at the beginning it was so neat and tight. Pushing the concept towards SPA might be better off in a follow up pull request. I just tested the implementation and directly ran into errors where the page switch was broken:

[rodja]

About your comment about full-page reevaluation vs partial execution:

Every single time you re-navigate using soft reload, it re-executes [everything]

While this can be nice in some circumstances, it can also be very problematic in others. For example if you have a costly database request in the outer layout. Or a search filter which should not be reset. In short: re-execution of the whole page might be costly. It depends on the scenario. I suggest to

1. get "faster HTTP Reconnect" out of the door (eg. revert this PR to 2549b1c, before adding push/popstate)
2. hone and polish ui.sub_pages
3. (maybe) optimize page loads by building upon "faster HTTP Reconnect" and adding push/pop state + sending only the changed parts

[evnchn]

First of all, this PR is evolving so quickly! I was cleaning my room, and it took me a solid 15 minutes to re-familarize myself with this PR. Now, we need a summary.

• Originally, Project BASE (Because Asked Server Explicitly) was designed to target Project ACID (Automatic Client-ID) aka "software rugpull" #4552 (Project ACID) which came before it, with the selling points of:
  • Fast page switching (implemented via softReload)
  • Zero Flash Of Unstyled Content as a baseline guarantee.
  • Fast reconnection in case client destroyed, because disconnected over the timeout duration / server reboot (STILL to be implemented 😅)
• But then, the "Fast page switching" part happens to coincide with the Single-Page Application thing we are trying to make
  • Since I tapped into pushstate and popstate in 783cc34
• The concern of repeatedly sending constant elements between pages came, and Project REBASE was born, to deduplicate elements sent, so as to leverage the elements already in memory
  • First generation, only elements which are an exact match would be deduplicated. 18f83e7
  • Second generation, it was done on a key by key basis. 21c2fad

---

@rodja, I am so sorry for shipping broken commit in 21c2fad, leading to a general bad impression of Project BASE.

I was too happy at looking at my key-diffing code serving small payloads, that the page was broken and I did not realize.

Apparently, (1) there must have been some problem with the logic which diffs the elements on a key-by-key basis, in a sense that old keys are not dropped, and (2) even if I fixed it, some elements are still messed up, since Vue doesn't force a re-render, and so the old element sticks around.

Check the below, where I made sure to manually compare the elements:

JSON-wise, key 97 is fine. But, c97 is still messed up

So, we really can't proceed with key-level diffing, unless we dig deeper into Vue, which we need to get #4828 done first, in order to be in a better headspace to solve these kinds of issue.

But, regardless, assuming we do the commit before that, we still get project REBASE with SPA-like functionality.

---

For your suggestions:

1. Get "faster HTTP Reconnect" out of the door (eg. revert this PR to 2549b1c, before adding push/popstate)

I would then need to achieve the "Fast reconnection in case client destroyed, because disconnected over the timeout duration / server reboot (STILL to be implemented 😅)" which I have dragged on.

2. hone and polish ui.sub_pages

Sure, but I think you can handle it, right?

3. (maybe) optimize page loads by building upon "faster HTTP Reconnect" and adding push/pop state + sending only the changed parts

But I think this part is what the PR truly shines, and I am not sure if it would be better to finalize this before we ship the feature. Otherwise, I feel like softReload is just a function sitting in memory that is only used when the client is disconnected for a long time, making the utilization very low...

[evnchn]

3. (maybe) optimize page loads by building upon "faster HTTP Reconnect" and adding push/pop state + sending only the changed parts

Moreover, as we can see after reverting 21c2fad, push/pop state + sending only the changed parts already works on documentation.

So on that front, I'm not sure what work we have to do, other than pytest and documentation.

[rodja]

Ah ok. Only reverting the key-diffing but keeping pushState/popState might also be a good cut. But still I can break the documentation page quite easily. For example if you open / and then click on "Features" or "Demos" the page does not scroll. And after that when clicking on "Documentation" the page is white.

[evnchn]

Can you send me a reproduction?

Though, I'm thinking if we can't get it to work after enough attempts, that we remove the duplication removal, and just send the entire unmodified element dictionary over...

This is because, we have #4796 on our side, if we want to do deduplication as well. Moreover, the bugs we are seeing looks very similar to #4828, so it looks like it'd not be an easy problem to solve.

[rodja]

It seems that auto-scrolling works sometimes. But I found another, simpler reproduction:

Just repeatedly clicking on the logo shows a blank page.

I'm thinking if we can't get it to work after enough attempts, that we remove the duplication removal, and just send the entire unmodified element dictionary over...

Yes, the simpler this PR gets the better.

[evnchn]

Although I fixed @rodja's issue on repeated clicks, there is a big problem with this PR: memory leak in the browser-side

I have banged my head against the wall for an hour to no avail. If we do not address this, this PR cannot proceed.

I have tried:

• Clearing out the elements with mounted_app.elements = [].
  • Clearing out the events of all elements before doing that.
• Try ensure complete deletion of the old Vue app besides unmounting it.
• Take another approach by updating the elements of the existing Vue app.

All attempts came up dry and I am lost as to how I can fix it, especially since the JavaScript-side is so complex...

---

At least I am pretty sure that #4821 won't suffer from this issue, since it doing things in the NiceGUI-layer.

[evnchn]

The plot thickens, as we see that there is no memory leak with a simple page (as seen by how the DOM nodes can come down naturally)