GSoC 2019 - Work Submission

Peter Bell

For the past 3 months I have been contributing to SciPy for my Google Summer of Code project “Revamp scipy.fftpack”. As originally stated, the main goal of this project has been to: “design and implement a backend interface which will allow different libraries to be called underneath the scipy.fftpack interface.” However, from discussions in the Community Bonding period, this project has grown into a new subpackage scipy.fft that is a complete rewrite of scipy.fftpack from the ground-up. scipy.fft features a new improved interface, makes use of a new FFT implementation and of course features the backend system discussed in my original proposal.

My most significant work on scipy.fft has been split over 4 different SciPy pull-requests (PRs), with many smaller SciPy PRs and contributions to other projects happening in-between. The first of these was merged in scipy#10238. This PR defines the scipy.fft interface and integrates the vendored pypocketfft library to implement the FFTs.[1] As part of this I also improved pypocketfft itself by changing the interface for transform normalisation and adding a complex FFT that is optimized for real input arrays. Here my edits to pypocketfft are separated into scipychanges.patch and scipy_features.h but in future pull-requests I contributed to pypocketfft directly so that updating pypocketfft within SciPy was made much simpler.

The second major PR was scipy#10383 which added backend support to scipy.fft. This is based on the uarray project which defines a protocol for dispatching function calls to different backends and provides a python library for creating uarray functions (multimethods). As part of this work I made several contributions to uarray itself (list here), the most significant of which was a complete rewrite of the uarray function call protocol into C++ using the python C-API. Bringing the function call overhead down by an order of magnitude to ~700ns per function call from ~5us. This change was merged as uarray#170. Related to this work was two scipy.fft backends that I have contributed to other FFT projects in pyFFTW#269 and cupy#2355.

Around the same time that my backend PR was merged, Martin Reinecke of pypocketfft informed me that he had expanded his library to include not just fourier transforms but also sine and cosine transforms. This lead to scipy#10493

in which I migrated scipy.fft over to these new transforms instead of the old FFTPACK transforms. With this, scipy.fft became a complete replacement for FFTPACK and I submitted scipy#10507 which both updated scipy.fftpack to remove the old fortran code entirely and updated scipy.fft to be completely independent of scipy.fftpack.

With my main project’s goals met, I also had time to work on a number of other enhancements to scipy.fft. For example, in scipy#10570 I replaced the next_fast_len implementation with a C++ version that makes use of a better search algorithm (pypocketfft!17) and takes into account the additional prime radices that are supported by the pypocketfft transforms. I have also worked on adding multi-threading to scipy.fft by contributing a custom thread-pool in pypocketfft!23. Support for this is added to SciPy in scipy#10614 which has not yet been merged.


Thanks to the SciPy community, especially my mentors Eric and Greg, for all their help and discussion surrounding the project. To Martin Reinecke for developing pypocketfft and being very responsive to suggestions and merge-requests. To Hameer Abbasi for his work on uarray. And finally to Google for running the GSoC program.

Summary of PRs






[1] Note that pypocketfft.h and pypocketfft.cxx are the work of Martin Reinecke.