PySwarms: a particle swarm optimization library in Python

For the past few weeks, I’ve started an open-source project in Python by building a research toolkit for Particle Swarm Optimization (PSO). PSO is a heuristic search algorithm that was inspired by the social dynamics of birds and bees. What made me interested in PSO is that it tries to simulate group behavior when faced with a certain objective, something that can also be observed in humans. Furthermore, the standard algorithm is simple, and can be used in a variety of applications.

• Documentation: https://pyswarms.readthedocs.io/en/latest/
• Github Page: https://github.com/ljvmiranda921/pyswarms
• Update: We were published in the Journal of Open Source Software (JOSS)!

Note: We had major API changes since this post. Some code snippets might not work as intended anymore. I suggest you look into the official documentation and the Github page for the current API

Inspiration

If one is to look at published literature, there is a multitude of variations in the standard PSO algorithm, all of these being applied in different situations. It then becomes hard for researchers to benchmark their results because there’s no unified framework to do it. Hopefully, a standard library should exist for implementing not only the classic PSO algorithms, but also the state-of-the-art variations in literature.

In addition, I found some of the current implementations a bit lacking. tisimst’s work, although useful, is not extensible and works only on one type of problem (although I named my project with respect to theirs). On the other hand PyGMO and DEAP, although mature, are more generalized to evolutionary computation algorithms.

Features

I realized that if I am to build an optimization library, it must have three important features: (1) the user-facing API must be easy-to-use, that is, an optimization can already be done in less than 5-8 lines of code, (2) supporting utilities must be included to assess optimizer performance, and (3) the programming API must be highly-extensible. These three are the guiding principles when I built PySwarms.

Easy-to-use optimization tool

This feature is highly important especially for users who just wanted a quick-and-easy optimization process. I hope that this can minimize “frustration” in setting up various configurables and the like. For example, if we want to find the minima of a sphere function using global-best PSO, we only need to import the pyswarms.single.GlobalBestPSO class and the built-in pyswarms.utils.functions.single_obj module to do optimization:

# Import PySwarms
import pyswarms as ps
from pyswarms.utils.functions import single_obj as fx

# Set-up hyperparameters
options = {'c1': 0.5, 'c2': 0.3, 'w':0.9}
# Call instance of PSO
optimizer = ps.single.GlobalBestPSO(n_particles=10, dimensions=2, options=options)
# Perform optimization
cost, pos = optimizer.optimize(fx.sphere_func, print_step=100, iters=1000, verbose=2)

Excluding the comments and spaces, it only takes five lines to perform optimization. Compare this to DEAP’s implementation. Although in all fairness, DEAP can be seen as a lower-level computational tool than what PySwarms is aspiring to be.

More examples can be seen at the documentation.

Supporting utilities to assess optimizer performance

Enhancements such as support-utilties are very important in assessing swarm behavior and optimizer performance. As of now, a PlotEnvironment class is implemented to perform such tasks. The idea is to pass the optimizer in this environment in order to call various methods such as plot_cost() or plot_particles2D(). The visualization tool is built on top of Matplotlib in order to provide deeper customizability to the plots.

However, the default settings are already decent. Consider the same example as above as we pass it in the plot environment:

# Import PySwarms
import pyswarms as ps
from pyswarms.utils.functions import single_obj as fx
from pyswarms.utils.environments import PlotEnvironment

# Create optimizer
options = {'c1':0.5, 'c2':0.3, 'w':0.9}
optimizer = ps.single.GlobalBestPSO(n_particles=10, dimensions=3, options=options)

# Pass to plot environment
plt_env = PlotEnvironment(optimizer, fx.sphere_func, 1000)

From this we can call various methods already. If we call plot_cost(), then we can obtain a plot similar to the one below

Figure 1: Calling the plot_cost() method generates a cost history line plot.

More so, we can even animate swarm movement. Recall that we are using a Global-best PSO algorithm, where each particle compares itself with the best-performing particle in the group. Notice in the figure below how these particles converge on the global-best at position (0,0,0). To generate a 3-d animation, we just need to call the plot_particles3D() method.

Figure 2: Particle movement generated by calling the plot_particles3D() method.

Highly-extensible API

I think that it is important to build an easy to use API for researchers an contributors who wanted to implement their own techniques or add new features. Honestly, this has become a challenge for myself, being new in the open-source field. I’m still iterating my base classes but I’m confident that most of these are now extensible as I’ve imagined.

The main idea for this is to inherit from base classes in order to implement new techniques. Most of the development time goes to the base classes, making sure that they are applicable in the most general sense. These include an array of getters, attributes, and abstract methods that I deem universal in most PSO implementations. Thus, if one is to observe the implementations on pyswarms.single.GlobalBestPSO and pyswarms.single.LocalBestPSO, not much redundancy exists between the two.

The main idea for implementing an optimizer can be seen in this guide. You can also view the API documentation here.

Tutorials

I wrote some accompanying tutorials on using PySwarms, most of these include use-cases on where to use Particle Swarm Optimization in some problems, and how PySwarms can make the implementations much easier. They are all in the documentation, but I will list some of them here:

• Basic Optimization
• Training a neural network classifier
• Feature subset selection

For Developers

If you wish to contribute on PySwarms, simply check the instructions in this link. In case you are interested and found this project nice, please leave a star on GitHub!