Coding The Volume Viewer

You may notice that around this site, I have added an interactive 3D volume explorer for manipulating MEP data. This is actually quite an impressive and snappy little addition to the site, and one that I hope helps make obvious the advantages of 3D imaging. The data I have posted so far has all been publicly available for some time, but it has never been this straightforward for someone to just click a single link and immediately explore it. I’m quite pleased with how this came out.

And yet, I have basically no idea how it works. It’s written in JavaScript, and I have never written any JS before. It’s backed by vtk.js, a lovely piece of software from our friends at Kitware; while I made the executive decision to implement it in vtk.js, I have never written any VTK before. This was entirely coded by AI, over the course of a weekend. The initial seeds of this were written by a frontier model¹, but I find Claudish to be annoying, and the free plan is rather limited both in terms of how much code it can write before hitting the daily limit and how clumsy it is to debug code when the AI can’t access and edit files directly on your system.

So, after the initial prototype I switched over to my new favorite toy: an agentic coding tool running with the Qwen 27B open weight model on my own computer². Since I have been doing computational microscopy since before the AI boom, I was lucky to have a nice collection of GPUs in my workstation already. With enough VRAM at your disposal, these smaller models are actually surprisingly capable. Whatever deficiencies in thinking they may have are easily made up for by the fact that I can simply let them think as long as I please, with no usage limits to worry about.

The setup took some tuning, to balance speed vs context size and other factors, but now I have a faithful coding companion that costs me nothing (other than making my office warmer). Figuring out this tuning was challenging, as the people who like to write about these things online don’t seem to have any actual use for this stuff beyond comparing contrived benchmarks, and so it was far from useful right at the get go. There are a million knobs to tweak but it seems to boil down to this: for the model to do complex tasks it needs a large context window, which takes a lot of VRAM. There is no free lunch, and this is why HBM is the hottest commodity in the world right now. There does seem to be a crossover point, at around 48 GB of VRAM, where the Qwen models become noticeably more capable at agentic coding. The volume viewer was made on a single 24GB GPU–its comments are full of gibberish, and it took countless rounds of prompting to implement any features, but in the end it worked. Part of this difficulty is likely due to the fact that my setup did not give the model the ability to search the web, which meant it had no access to the documentation or example code for vtk.js. To figure out how to do things, it had to instead download the minified JS and work entirely from the (stripped, uncommented) source code.

For “real” work I still keep myself intensely in the loop. The simpler models are not quite capable of being given a vague direction and then proceeding on their own to produce magic; they need you to do some thinking and checking and prodding for them. But they are a great cure for a certain type of writers block, where you know the approach but haven’t mustered the energy (or found the time) to hammer out the details. Just describe your thinking, and let it churn. I find lately that I am a much better editor than drafter, in general, so this has unlocked a logjam for me. I don’t know how graduate students will learn to produce original work in this field anymore, but having reached the stage of being more of an orchestrator of talent I am shocked at what the robots can do with a little encouragement.