Quick Tech Tutorial - How to Print FastAI Training Progress to a Streamlit App

👋 - quick tutorial here. The title is self-explanatory, so let me know if you have any questions.

We're building an application in Streamlit right now that does everything from defining a data loader, training multiple traditional ML models, and training a deep neural network. I enjoy using FastAI for projects like this, but the library hides a lot under the hood and I wanted to see the network training progress in the GUI.

Most suggestions to get the progress into the Streamlit app were some version of this:

    # Create a string buffer and redirect stdout to it
    buf = io.StringIO()
    with contextlib.redirect_stdout(buf):
        learn.fit_one_cycle(options["epochs"])

    # Get the stdout string from the buffer and print it with st.write()
    output = buf.getvalue()
    st.write(output)

That code snippet will essentially just collect the output and will spit it out after the training is done... which isn't an awesome replacement for a progress dialogue.

Here is the alternative that I put together:

Just punch that class directly into your learner callbacks like this:

learn = Learner(dls, model, opt_func=Adam, loss_func=L1LossFlat(), lr=defaults.lr, cbs=[SaveModelCallback(monitor='valid_loss', fname='my_model'), StreamlitProgressCallback()])

and track your training progress!

I just threw this together for a project, so please ping me at caleb@depotanalytics.co if you have any questions or comment on the public gist with suggestions for cleanup and improvements!