How to Add a Progress Bar for Matlab parfor Loops

Posted in Simulations / Tools
How to Add a Progress Bar for Matlab parfor Loops

An easy way to track progress in parallel for loops in Matlab

Running parallelized Monte Carlo simulations in Matlab is easy with parfor loops. But there’s a catch: how can you know how much of the work is done and how much is left?

This post is about an efficient and easy visual way to keep track of progress in parfor loop (and some less obvious Matlab features I learned about when trying to figure it out).

Contents

  1. Introduction
  2. The Challenge with Tracking parfor Progress
  3. The Solution: A Parallel Progress Bar Function (by Example)
  4. Going Deeper: How Does createParallelProgressBar Work?
    1. DataQueue and Worker Communication
    2. Creating and Initializing the Waitbar
    3. Using Persistent Variables for State Tracking
    4. Updating and Closing the Waitbar with a Nested Function
    5. Attaching a Listener to the DataQueue
    6. Customizing the Progress Bar

The Challenge: Tracking parfor Progress

Tracking progress in a regular for loop is straightforward because each iteration runs sequentially. Simply dividing the current iteration number by the total number of iterations gives an accurate estimate of progress.

However, parfor loops work differently. Matlab uses dynamic scheduling to assign iterations to workers as they finish their previous tasks. This means that iterations are not completed in a simple (or even a deterministic!) order. As a result, iteration numbers alone do not reflect total progress accurately.

The Solution: A Parallel Progress Bar Function

The solution? A visual progress bar that uses the messages each worker sends back to the central process! Matlab’s parallel.pool.DataQueue allows us to listen to such messages and gather progress updates.

With this approach, the createParallelProgressBar function tracks parfor progress with just two lines of extra code in the script.

Here’s how to use createParallelProgressBar in a parallelized loop:

  1. Download or copy the createParallelProgressBar function and place it somewhere along your Matlab path.
  2. Insert into your code the two lines marked with <---, as shown in the example.

An example:

numSamples = 100;

% Initialize the progress bar
queue = createParallelProgressBar(numSamples);  %  <---

parfor iterID = 1:numSamples
    % Simulate computation, replace with your own
    pause(rand(1));   
    % Update progress bar after simulation is done
    send(queue, iterID);  % <---
end

The code for createParallelProgressBar is as follows:

function queue = createParallelProgressBar(totalIterations)
    % createParallelProgressBar Initializes a progress bar for parallel
    % computations with dynamic color changing from dark orange to blue.
    %
    % Args:
    %     totalIterations (int): Total number of iterations for the
    %                            progress bar.
    %
    % Returns:
    %     queue (parallel.pool.DataQueue): DataQueue to receive progress
    %                                      updates.
    %
    % Example usage in a parallel loop:
    %     numSamples = 100;
    %     % Create progress bar
    %     queue = createParallelProgressBar(numSamples);
    %     parfor i = 1:numSamples
    %         % Simulate computation
    %         pause(0.1);
    %         % Update progress bar
    %         send(queue, i);
    %     end

    % Initialize DataQueue and Progress Bar
    queue = parallel.pool.DataQueue;
    progressBar = waitbar(0, 'Processing...', 'Name', 'Computation Progress');
    
    % Access the Java-based components of the waitbar
    barChildren = allchild(progressBar);
    javaProgressBar = barChildren(1).JavaPeer;  % Access the Java progress bar

    % Enable string painting to show percentage inside the bar
    javaProgressBar.setStringPainted(true);

    % Reset persistent variable count
    persistent count
    count = 0;

    % Define colors between which the bar interpolates
    colorEnd = [12, 123, 220] / 255; % light blue
    colorStart = [171, 94, 0] / 255; % brown-orange

    % Nested function to update progress and color
    function updateProgress(~)
        count = count + 1;
        shareComplete = count / totalIterations;
        
        % Update waitbar position
        waitbar(shareComplete, progressBar);

        % Calculate color transition
        currentColor = (1 - shareComplete) * colorStart + ...
                        shareComplete * colorEnd;
        red = currentColor(1);
        green = currentColor(2);
        blue = currentColor(3);

        % Convert RGB triplet to Java Color
        javaColor = java.awt.Color(red, green, blue);

        % Set the progress bar color
        javaProgressBar.setForeground(javaColor);
        
        % Close progress bar when complete
        if count == totalIterations
            close(progressBar);
            count = [];
        end
    end

    % Add listener to the DataQueue
    afterEach(queue, @updateProgress);
end

The solution is based on the template given in the official documentation for parallel.pool.DataQueue. It adds some nice features, both aesthetic (explicit percentage progression, changing colors) and more substantial (automatic clean-up after completion). Tested with Matlab 2024b on Windows.

Going Deeper: How Does createParallelProgressBar Work?

How does this progress bar work? Overall, the flow is:

  • Initialize: createParallelProgressBar initializes a DataQueue, waitbar, and a persistent count variable.
  • Create Listener: It attaches the updateProgress function as a listener to the DataQueue, so that each message from the workers will trigger a progress update.
  • Run in Parallel: As the parfor loop runs, each worker completes iterations, sending messages to the DataQueue.
  • Update Progress Bar: Each message received by DataQueue triggers updateProgress, incrementing count and updating the waitbar to show real-time progress.
  • Complete and Close: When all iterations are done, count matches totalIterations, and the waitbar is closed automatically.

Going a little deeper into each component here:

1. DataQueue and Worker Communication

The DataQueue object enables communication between parfor workers and the main process. Each worker sends a message to DataQueue when it completes an iteration, allowing us to track progress:

queue = parallel.pool.DataQueue;

Each time a worker completes a loop iteration, we will send a message to the DataQueue in the main file.

2. Creating and Initializing the Waitbar

waitbar is a function that creates a pop-up window with a horizontal wait bar. It is initialized with 0 progress made, some message ('Processing...'), and a title ('Computation Progress'):

progressBar = waitbar(0, 'Processing...', 'Name', 'Computation Progress');

3. Using Persistent Variables for State Tracking

We use a persistent variable to keep track of how many iterations have been completed up to a given moment. persistent variables retain their value across multiple calls to the function. Unlike global variables, such variables are local to the function in which they are declared.

Specifically, we declare a persistent variable count:

persistent count
count = 0;

We initialize count to 0 and increment it by 1 each time an iteration is completed.

4. Updating and Closing the Waitbar with a Nested Function

The nested function updateProgress is responsible for is called each time a message is received by the DataQueue from a worker.

function updateProgress(~)
    count = count + 1;
    shareComplete = count / totalIterations;
    
    % Update waitbar position
    waitbar(shareComplete, progressBar);

    % Omitted color computations, see below
    % <..>  
        
    % Close progress bar when complete
    if count == totalIterations
        close(progressBar);
        count = [];
    end
end

Every time updateProgress is called, it increments count by 1. Then the progress on waitbar is updated based on the new value of count.

Once count reaches totalIterations, all tasks are complete. Then the waitbar is closed, and count is reset.

5. Attaching a Listener to the DataQueue

We need to make sure that the updateProgress is called every time an iteration is finished. We create a suitable listener:

afterEach(queue, @updateProgress);

To make use of this listener, we insert the following sender line in the main script

send(queue, i)

Effectively, every time a worker completes an iteration and sends a message via send(queue, i), Matlab calls updateProgress in the main session to update the progress bar.

6. Customizing the Progress Bar

MATLAB’s built-in waitbar function has a very limited set of options for styling. To change the bar color with progress, we access the underlying Java JProgressBar that MATLAB uses to render waitbar:

    barChildren = allchild(progressBar);
    javaProgressBar = barChildren(1).JavaPeer;

The line javaProgressBar = wbc(1).JavaPeer retrieves the Java-based waitbar component which we can then modify.

    javaProgressBar.setStringPainted(true);

This line enables the display of a percentage or other string inside the progress bar. With this line, we can then manipulate the color of the bar by passing an RGB triplet using java.awt.Color:

    javaColor = java.awt.Color(red, green, blue);
    javaProgressBar.setForeground(javaColor);

The color transition goes from dark orange to light blue as progress moves from 0% to 100%, with the colors chosen to be as accessible as possible.

Conclusion

With createParallelProgressBar, you can easily track progress in parfor loops. Using DataQueue for worker communication and accessing Java properties for color customization gives a lightweight and visually informative solution.

© 2024. All rights reserved.