EEGStudyFlow is the name I use for the workflow patterns that sit around an EEG analysis project: where raw recordings live, how preprocessing state is tracked, how quality-control decisions are recorded, and how analysis-ready outputs are handed off. This page is intentionally brief for now, but the project goal is simple: make EEG studies easier to reproduce without asking every researcher to keep the whole pipeline in their head.
In April 2025 I had the idea to get PyLossless put onto the DRAC clusters as an official module because I needed to clean at least half a dozen different datasets. At the time, I managed a lot of my projects with virtual environments per project, so this was going to involve reinstalling the same package over and over. Funnily enough, the module was already there! When I requested it be updated by the internal team, they said they had a few things that they’d like changed about it, especially if I was going to be doing active development on it myself. ...
In the early days of working for SHARCNET, my colleague and I decided to standardize how cluster metrics were computed across our internal data frames. As mentioned in a previous post, part of the solution was pandas. The second part was figuring out how to deploy the package for others to contribute to, as well as install on their own specific HPC clusters. Some quick searching revealed that PyPI and pip were the way to go. ...
Coming from a slightly different angle this time, I found that researchers were often isolating themselves to fewer resources on HPC systems by not investigating what the node feature mixture looked like. As such, this talk was created to help direct potentially abstract development efforts toward optimizing for the feature sets that are most available on an HPC cluster. Below is my abstract for the talk as well as the recording: ...
Eventually I got to the point in data analytics where keeping things in lists, or lists of lists, was no longer quite cutting it. My processing was slowly starting to grind to a halt, and things were getting way too abstract. I decided to call up a friend who had worked in the business longer than me, and they suggested “pandas.” I was vaguely familiar with it because users and clients had used it in the past. A “DataFrame” did sound like it would take care of a lot of my problems after reading the documentation casually… ...
Back when I first got hired at SHARCNET, I used a lot of Python. I mean a lot. This meant I quickly became the lightning rod for all Python-related questions and commentary. During a fun Friday chat, a colleague remarked that Python was, on average, 40x slower than C++. I defended my current language of choice, saying it was better than that, surely. To make a long story short, I was wrong. It really is about 40x slower depending on the problem. Determined to prove myself capable, and my language of choice a bit more defensible, I decided to look into ways to make Python faster. ...