Speed-testing @stlite/desktop
At Depot Analytics, we often are developing applications and systems that hold proprietary information - anything from sensitive personal information to trade secrets.
So sending that data to hosted servers is hazardous at worst and suboptimal cost efficiency for our clients at best.
The ideal solution then (so long as the compute requirements are minimal) is to build a desktop app or a locally networked app for the client. Their network, their security measures, their controls.
But what about when we're using Python? You can read about Python GUI frameworks (PyQt, Tkinter, Beeware, etc.) elsewhere, but they essentially all either look awful or are nightmarish to use.
Enter Streamlit for desktop apps.
@stlite/desktop essentially turns your Streamlit Python app into an electron/JS-driven desktop app. All the Python gets translated to WebAssembly and executed by your browser - no need for any local Python package installs or server-side code. Crazy stuff.
But how slow is it compared to running Python directly?
Rendering the GUI
Note that this is not testing the longest portion (seemingly) of loading the Desktop app, which is when the JS module is doing some sort of wizardry to make Python browser-runnable. That part takes 3-5 seconds. Just about every time.
This is the basic rendering test:
if __name__ == "__main__":
st.write("Speed test app")
# do a mathematical stress test with numpy and pandas
# create a dataframe
df = pd.DataFrame(
np.random.randn(200, 3),
columns=['a', 'b', 'c'])
# create a line chart
chart_data = pd.DataFrame(
np.random.randn(20, 3),
columns=['a', 'b', 'c'])
st.line_chart(chart_data)
# create a map around Tenerife
map_data = pd.DataFrame(
np.random.randn(1000, 2) / [20, 20] + [28.2916, -16.6291],
columns=['lat', 'lon'])
st.map(map_data)
end_render_load_time = time.time()
st.write(f"Rendering a basic GUI in: {end_render_load_time - begin_render_load_time}")
Running this with the normal Python approach yields this result:
0.52 seconds.
Running via the Desktop app give this performance:
The Desktop app actually renders faster - 0.29 seconds. My guess is that the "quicker" speed here has something to do with the loading portion that is trickier to time directly.,
Pure Computation Tests
I then created two tests: one that runs a pyodide library (numpy) that has been translated from its C backend to be compatible with stlite, and one test that runs pure Python computations. (** The tests are not meant to be the same runtime/computation intensity/etc - just benchmarks to run on both streamlit and stlite, then compared.)
def pyodide_compute_test(num_iterations):
"""do a mathematical stress test with numpy"""
# setup a timer
start_time = time.time()
# do 10000 calculations with numpy
for i in range(num_iterations):
np.random.randn(100, 100)
end_time = time.time()
# add a row to test_df
append_csv('pyodide_test.csv', num_iterations, end_time - start_time)
return
def pure_python_compute_test(num_iterations):
"""do a mathematical stress test in pure python"""
start_time = time.time()
for i in range(num_iterations):
arbitrary_list = [x * 2 for x in range(100)]
end_time = time.time()
# add a row to test_df
append_csv('pure_python_test.csv', num_iterations, end_time - start_time)
return
and created a GUI for them
num_iterations = st.slider("Number of numpy rand computations to do", 1000, 100000, step=1000)
# add a button to run the computational test and get return value
st.button("Run pyodide library compute test", on_click=pyodide_compute_test, args=(num_iterations,))
# dynamically display the global variable test_return_str, so that it updates when the variable is updated
st.write(pd.read_csv('pyodide_test.csv', index_col=0) if Path('pyodide_test.csv').exists() else None)
st.button("Run pure python compute test", on_click=pure_python_compute_test, args=(num_iterations,))
st.write(pd.read_csv('pure_python_test.csv', index_col=0) if Path('pure_python_test.csv').exists() else None)
Running on Streamlit via normal Python, the tests yielded these results:
and via the Desktop app:
So the speed factor plots of Desktop (browser-run)/Python direct look roughly like these
Pure Python
Pyodide Libraries
I'm sure that there are better tests to run than these, but frankly I was pretty surprised that the pyodide libraries performed so much better than just pure Python calculations (in relative terms of course).
Conclusion
The slowdown from Streamlit when moving to desktop is likely somewhere from 2-5x depending on the type of application that you're running.
Notably in my case, the GUI still feels instantaneous while executing as a Desktop app. Reasonable computations are still really quick. Taking a slowdown like this into account only really matters when doing significant amounts of low-latency compute.
Comments