Why Build Things?

Build things that make the impossible possible (not to make the easy thing easier).

This idea is shamelessly stolen from Jon Blow, an independent game designer. He stated this in the context of programming language design, but in my opinion it’s broadly applicable. One example he gives is

LLVM would be a good example of this. By decoupling programming language from generated binary via a intermediate representation, it made it possible to develop new programming languages without needing to build the whole toolchain; making languages like Rust, Zig and possible (even if they only use LLVM to get started).

At Pneumatica, this philosophy sits underneath our software stack. We have a custom modelling suite that makes it possible to re-use experimental data from different biological experiments when parameterising models, which is built on ideas from systems-theory I’ve been working on for about 10 years now.

If it’s core to your business, build it (otherwise buy > outsource > build).

Part of the proposal for starting a tech business is that you have particular insight into your customers needs that other companies don’t. If there is a product on the market that you can buy that solves their problem, then you’re wrong and have made a strategic mistake.

Nobody has the resources to build everything so one has to make trade somewhere. In my opinion, it’s crucial to have complete ownership and control over the IP base of your business – hence build. If you own it, and have built it, you have the knowledge and capacity to respond to feature requests and bug fixes on a time line you control – not the case if you need to ask a 3rd party to fix issues for your customer.

Similar principal applies if you are selling services, not software. It’s just that your ‘customers’ are internal.
For everything else – especially stuff that doesn’t need to change frequently – buying is usually a better bet, as you pay either a one-off fixed cost, or a small recurring one, and don’t need to allocate precious effort to developing and maintaining peripheral products.

Use time spent – not necessarily pain – as the trigger.

I think this is one of the traps I, like many software people, have hit in the past. There is a part of the product that sucks to work on, and every time you touch it you feel frustrated. However, the fact that it’s a pain is not sufficient reason to focus any significant effort on fixing it. Instead, it’s the tasks that you find having to spend significant time on, be it administration or data-wrangling; anything that requires you to spend time doing low-value work, baby-sitting or repetitive tasks are ideal candidates for building a an automated solution.

Our robotics development fits into this category. Running cell experiments (particular the ones we like to use) is a time-consuming, labor intensive activity. Right now there isn’t a “PCBWAY” for cell biology (i.e. cell experiments full automated and on-demand) so we’re building a small scale one ourselves until someone else comes along with a better solution.

Automate the places where mistakes matter.

I’ve been a strong advocate for continuous testing for quite a while now, motivated by the fact that I know I (and every one else) miss things during code review. Even better is if your automated tests run end-to-end on the real hardware. One of the first things I did when I started at Arkeus was brought in a robust continuous testing policy, which was complicated by the fact that some of Arkeus products at the time needed specific hardware platforms to test the software stack.

It’s super important to get this right with hardware/software systems. “It crashed” is recoverable if you’re building a desktop application, but “it crashed” is harder to stomach if your computer is hanging off the side of an aircraft in flight. Logic bugs in firmware have cost space companies millions of dollars, and even some astronauts lives. For us, getting it wrong means burning out motors and frying electronics due to software mistakes, or contaminating experiments (potienally wasting many weeks of work) due to mistakes in handling biological materials.

The latter is another reason why we’re pushing ahead with robotics as part of our development.

Do “scaling enablers” only when you actually need it.

I think every software person understands this; “don’t duplicate” is drilled into us during university.
It’s always tempting to “make that piece of code a library” to make it reusable – and hence scalable.
Maybe there is part of the experimental process (such as preparing media) that you could probably automate

There is big risk in doing that too early, before there is a case for scaling, as it costs time and effort that are better off spent solving more important problems. It takes experience and/or clear demand to know when that’s the right call. If it is the right call, you probably have already hit it via the first point. I.e. scaling can be impossible without the right tool, motivating the effort to build them.

What I Learnt About Deep Tech After Academia

This follows on from The Category Error in Deep Tech. That post was about the difference between a tech company and a deep-tech company. This one is more for my academic friends thinking about the jump: what I found different once the work had customers, contracts and delivery dates attached to it.

Academia and business are both hard, just in different ways.

Research asks whether something is possible or useful. A business has to make it work for someone who will pay for it, on a real schedule. In deep tech, those get mixed together.

I learnt a lot of this as a software architect at both UAMTec and Arkeus, in the company’s most senior software role at the time, reporting directly to the C-suite. My work sat between the technical team, product and customer commitments.

A mission does not pay the bills

In academia, it is normal to justify work by its possible contribution to science or society. That makes sense for research. In business it becomes dangerous when it replaces the commercial premise.

A business can absolutely do useful things for science or society. But it has to survive long enough to do them. That means solving a problem for people who both can and will pay, and (importantly) investing significant time and effort in the development of those relationships.

One thing I think both Arkeus and UAMTec did well was actually name and build relationships with their key first customers – not just general “defense” or general “water”. This allowed them to tailor their delivery to actually secure sales, whilst being careful not to over-specialise on those one customers needs.

I think this is probably one of the bigger paradigm shifts that the academic-to-business move requires. That is, understanding you must be able to articulate exactly who you are serving (the customer), what they need (the problem) and how you address that need (the product).

Deliver the thing that can be delivered

Most academics know that academic code is different from production code. Closing that gap combines technical work with the need to deliver on a particular date. At Arkeus, the software ran in a time-critical, resource-constrained environment, and missing the project deadline was not an option. That technical complexity and delivery pressure was what made the scope and design decisions matter.

The project also had to build a product and technology base that would give the company a real long-term competitive advantage. Delivering the contract alone would have left us doing contract work with a more interesting R&D story attached.

That is where I developed a fairly strong view about scope. For the immediate delivery, we had to cut high-risk, low-TRL(technology readiness level) features and focus on what we could actually ship against the customer requirements. I worked closely with the product owner to keep that work connected to both the customer’s immediate needs and the written project requirements.

In many cases, design can support rapid delivery while creating affordances for later capabilities. That means making the relevant seams available without building the later capabilities early. Avoiding speculative work and a general-purpose framework still leaves room to choose an architecture where adding a feature later does not mean pulling apart the thing you had to ship now. The customer cannot buy the capabilities you might have in two years, but the decisions in the first delivery determine how hard they will be to build later.

Middle-out: connect the short term to the long term

The first product has to be small enough to build, but still point at the company you are trying to become.

The top-down plan starts with the long-term technology that will make the company hard to replace. The bottom-up plan starts with a customer’s immediate problem and asks what can be delivered credibly now. Deep tech needs to nail this middle-out: work backwards from the customer, and forwards from the differentiating technology, until the two meet in a useful first product.

Without that, you either try to build the whole vision before anyone can buy anything (which is generally not feasible), or you cut so much that you are just another commodity supplier. It also forces a deliberate decision about what is deferred, what must be retained, and what would quietly close off the path to a real advantage.

I think Arkeus handled that well. We kept cutting the immediate problem down until it was shippable, but consciously and explicitly laid the groundwork for integrating revolutionary features as they came online. That seems to have paid off.

Customers start with their problem

Researchers often start with a capability: “what could this unlock?”, but in the commercial setting customers usually come with something already on fire: a deadline, a cost problem, a risk they need off their desk. It’s not that companies don’t have long term goals, it’s that they have a hierarchy of needs which means the deals close easiest when only when the you can explain what it fixes now, what it costs, what can go wrong, and why the customer should take the risk.

I have sat in project updates where an issue threatened a contract deliverable. In those meetings, novelty mattered only if the proposed technology could solve the problem in time, for a plausible price, without creating a larger problem somewhere else.

Even if your product does solve a clear and obvious problem, a product claim needs evidence. A serious customer will compare it with what they already use, a competitor, or some independent measure. At Arkeus, that meant comparing new localisation approaches with GPS, running camera shootouts, and evaluating different single-board computers. Those comparisons gave the customer a way to decide whether the claim was real. A product with no obvious comparison, trial or benchmark will struggle to be taken seriously.

Most organisations, especially large ones, adopt new technology slowly though sometimes there are smaller units with enough budget, authority and pain to try something new. Some commercial contexts have an obvious early customer; others do not. Where they exist, finding them is much better than trying to sell the whole market at once.

This leads into the final point.

Sales is work

I saw another side of this at UAMTec, where I was brought in as the senior software person to work on autonomous submarines. The company already had BarrelEye, a diving product that was gaining adoption.

I had not seen the amount of work behind that adoption. The business-development and sales lead spent a great deal of time finding potential customers, following up by phone and email, answering questions, and keeping conversations alive. It was a useful reality check for me: adoption depends on the slow work of finding people, following up, answering questions and keeping the conversation going.

Arkeus did this well too. The executive team made good use of existing networks and built new ones to get the company in front of the right people. Even in the build phase, that was a hell of a lot of work, and it was in a different league from the networking academics are usually expected to do.

Key Takeaways

If i were to distill this down for a younger version of me, I’d summarise this as follows

As fast as you can, move to thinking about solving specific people/companies problems
(whilst still keeping the long term vision in tact)
Whatever you are doing, ship it as fast as reasonable
(whilst designing in from the get go, the piece to bring your vision to life)
Find a good business person, with good networks, and bring them on early.

Perspectives on AI at Pneumatica.

It took me some time to jump aboard the AI train. When chatGPT came out, I found it useful for certain “padding” tasks; like turning a set of dot-points into a position description but it just wasn’t mature enough to be part of my daily workflow.

It took me a couple of months to notice the Claude Code hype-train, and to overcome my initial skepticism to try it out. The fact that it was a command line application removed one of the biggest barriers for me; the web interface.

These days, i generally rely on OpenAI models with oh-my-pi as the harness. Here are some of my reflections on what’s worked and what hasn’t.

The Good

AI Agents as an shopping assistant for components.

When one builds a complicated machine like a robotic system, you usually have a bunch of constraints on your components; size, weight, power, cost, etc. I usually find this process incredibly time consuming; lots of searching, filtering and reading.
When building our current robotic system, i talked through requirements and sizing with my agent, then got it to search the web for options and rank the options based on our conversation history. This then either refined my thinking about the system design, eventually coming out with candidate parts. Having the agent dump the results to a spreadsheet, with technical specs and URLs to the data sheet, drawings and store is really, really helpful when building a bill of materials.

Hardware Abstraction Layer for Embedded peripherals.

A bit of background for those not familiar with embedded software engineering. It’s pretty common for embedded devices to consist of a microcontroller (uc) that talks to peripherals like stepper motor drivers, current sensors and other chips, connected by some standard (usually serial) interface. Microcontrollers are generally too small to host a traditional operating system (and having an OS like Linux often defeats the purpose; the goal here is to be small, fast, and close to the metal), but do tend to come with manufacturer support (or via 3rd party libraries) for configuring the microcontroller itself. For the peripherals, your often only have a datasheet and, if you’re lucky, a few hobbyist open-source partial implementations.

For our toolchain (rust). Using an agent to go from datasheet to binary interface bindings has saved me a ton of time an effort. The design often needs a few guided revisions, since agents are currently bad at architecture, but the basic functionality seems to be pretty good first go.

Doing the software implementation/refactor grunt work.

Sometimes I just need a quick python script to do X, that I’ll throw away afterwards; an agent is great for that. Sometimes I have a refactoring goal in mind, and I can easily express the intended outcome (even better if i can write the example as code); agentic coding is pretty good at moving the interface across, though it sometimes likes to take shortcuts. Sometimes I’m working on a new feature and I’ve got it to the point where the skeleton is there. AI can be pretty good at putting the meat on those bones.

“Rubber duck” debugging (for mech, elec and software)

I’ve worked with some pretty great teams in my time, and one thing they have had in common is a culture of sharing the gnarly problems, seeking an outside perspective, or even just someone to talk through the issue (which helps clarify the problem and solution).

Currently, we have a engineering team of 1, so agents have been a useful (but not always reliable) stand-in. It’s helped with walking through gnarly mech, elec and software problems, even if it’s not always a reliable substitute for a human colleague on the other end.

The Bad

Poor agent usage (eg. vibe coding) undermines the intuition -> expertise pipeline.

It’s very rarely the case that you know enough about the problem before you start building, or that the understanding is implicit-unconscious, so a big part of the first draft is learning the shape of the problem, a shape of the solution, and how to articulate what that looks like (in code, or design) using the tools of the domain. As you do this, you’re constructing a map of the problem-solution space for you to come back to later, and accumulating learnable experiences. Hand that first draft to an agent instead of doing it yourself, and you skip the very process that builds the map in the first place.

This is especially the case when you’re building something actually new – even if new is cobbled out of existing parts.

Agents currently suck at software architecture.

Software engineering is full of cargo-cults, fads, and baseless extrapolation. This is especially the case when it comes to architectural decision. For me, a software architecture is an organisation of information processing that needs to fit the constraints over the expected (bounded) time horizon of the a) business/intended use case, b) compute platform and/or operational environment, c) current (and expected future) developers background.

Architecture is as much cultural and organisational as it is technical; and a problem (i would argue) is not in the purview of current AI systems.

This part of this falls into a broader mistake that we make about cognition, and how we fail to recognise just how big the gap is between what we can clearly articulate, and what we intuitively understand. How when you work with people, our intuitions ‘rub-off’ on our colleagues (and vice-versa), which goes a long way to building an strong engineering culture. Obviously, you want to document this if possible, but a culture is a living-breathing thing, and current agents can only be a peripheral tool in that respect.

Tokenenomics

Not a surprise, but the opacity of token pricing, and how quickly the price can change. Everyone knows the pricing model isn’t a sustainable business, and I’m expecting some kind of rug-pull later this year (Anthropic already did around April, which is why I’m on GPT at the moment).
Regardless of whether the model does what you want, ties itself in circles, or deletes your file system, you’re still paying for the tokens. I’ve seen some decent results come of directing a team of agents, but then the cost skyrockets pretty quickly, and I’m just not sure the payoff is there.

Conclusion

AI agents as a genuinely good research assistant, datasheet-reader, and grunt-work machine, but a pretty mediocre architect that can easily flush money down the drain. I’m very hesitant about it’s ability to work on anything truly novel or niche, but am pretty happy for it to slap together a webpage and a quick-and-dirty script.

I worry mostly about finding the balance between the questionable/circumstantial short-term gains from heavy AI use, and the long term costs of opacity (both internally and externally) and technical/organisational/personal debt – which I think should not be underestimated.

3 Sacrifices for the Common Good

Like many of us, I always have a few projects on the go. Over the last year, I’ve mostly put them aside so that I can focus on Pneumatica.bio, but here I want to give a little overview on the goals, and where I got up to.

#1 Electronic Ducted Fan Quadcopter

I think ducted fans are cool, so I wanted to build a quadcopter based around them. I was thinking more of a drone for lifting stuff in complex environments. Fans mean no exposed spinning blades to chop my fingers on, and from what I read, fans do well in static thrust mode (i.e. mostly hover).

The frame is custom designed, and the flight control stack is based on the navio2 raspberry pi stack running ArduPilot. The drone has lifted off under it’s own power, but I hit a few challenges before I ran out of time to actually tune it.

In particular:

ArduPilot off the shelf is not at all friendly to the EDF configuration – in particular the PID loop needs a whole lot of tuning before it’s stable.
I am a very bad drone pilot. I’ve since been practicing in flight simulators.
I didn’t appreciate just how loud the fans are; way too loud to test inside my small townhouse.

So i have a few things left to do before I can really start to give this guy a spin, including building an outdoors test setup. One day I’d also love to put the decawave position system i have lying around on it.

I expect that once I actually fly it, I’ll realise the frame is not rigid enough, and the black PLA will struggle in the Australian sun. I’ve got some poly-carbonate that i think might work better (and am much more comfortable printing it than I was a year ago), but I also have some white PLA-AERO that i’m looking for an excuse to crack out.

#2 Hexapod

The hexapod project (which follows on from the previous darkpaw project), was/is quite an ambitious undertaking, and it’s now been going on for a few years. I had a number of goals with this project; to get good enough at CAD (specifically Fusion360), to teach myself the differential geometry approach to rigid body mechanics by reading and implementing A Mathematical Introduction to Robotic Manipulation by Murray, Li and Sastry, and to work out how to design software for rapid iteration of whole-body controller design. One of my goals was to use a whole-body feedback controller instead of the usual pre-programmed tripod gait that people implement.

Here is a video of it doing a test sequence.

Where I got to:

CAD is mostly done. I’m happy with the general design though the body needs a few more iterations that will be driven by the electronics housing needs.
Electronics. First pass is done. Robot moves off 12V 5A wall socket, which “should” be well within tolerance for the battery-base power supply, but I haven’t tested. On the plus side, i’ve more experience with power electronics via Pneumatica than I did initially so I’m less worried here.
‘Firmware‘ is right now a rust application running on a Raspberry Pi, with a proof-of-concept serial interface to the Dynamixl servo motors. It’s mostly scratch code for testing right now, but also the cheapest to iterate on once I’m happy.
Software-in-the-loop simulator. I have custom built simulator, written in rust and based on bevy and physx, that I’ll be using to test the whole body controller
Modelling & Controller design. Kinematic modelling is done, and uses the rigid body modelling toolkit i wrote as part of coker (more on this later). I’m still working through the whole-body controller and footstep planner and might release a technical note about the design later on.

#3 Bela Bass Synth

So not too long after I submitted my Ph.D. i treated myself to a new bass guitar from https://industrialradio.com.au/, which has a special fretboard design to output midi.
I’ve always been interested in digital signal processing so when I found out about Bela (which is a programmable real-time audio hat for the BeagleBoneBlack), i decided to buy one and try to build a bass synth for my midi bass. I was inspired by an old VST instrument called iBlit by ERS, which uses a synthesis approach called ‘Band Limited Impulse Trains’ (hence blit) originally from this paper.

This project has been through a few iterations. Currently:

The synthesis engine is working, but missing a few features
We have a web-app through which i can edit some patches.
We have 3D printed rackmount housing
I’m currently working on adding a touchscreen for patch editing, but having issues with the SPI interface.

Source code for this project is here. The current friction comes from the need to set up SPI signal diagnostics, but my electronics workbench is busy with Pneumatica robot stuff, so that’s parked for the foreseeable future.

The future

With Pneumatica in full swing, I probably won’t be spending too much time on these projects, and unfortunately they’re not really blocked by things a bunch of AI agents can fix.

The Category Error in Deep Tech

I’ve spent my career across the R&D spectrum, from pure research in applied mathematics through to embedded systems, control software, and now co-founding a biotech startup. That range has given me a clear view of one mistake I see made repeatedly, by founders, investors, and accelerators alike: treating “deep tech” and “tech” as the same category of company, and applying the same playbook to both.

A tech startup combines existing technologies to solve a problem. The risk is execution. A deep tech startup derives its competitive advantage from a technological capability that does not yet exist at commercial scale. The risk is the technology itself. That distinction changes everything. Deep tech sits at the intersection of research and business, and how you balance that tension is one of the most crucial things to get right. It’s also something that must be actively navigated.

The failure mode from the business side: pressure to ship an MVP before the core technology is ready produces a product without a genuine differentiator. Competitors catch up, or the company fails to deliver on the growth story it sold to backers.

The failure mode from the research side is subtler: founders commercialise their research without rigorously interrogating whether their particular technological advantage maps onto a large enough, hard enough problem. Great science and a viable business are not the same thing. If the problem you are solving has a solution by combining existing technology, you do not have a deep tech startup. You have a tech startup, and you should read the lean startup canon and race toward product-market fit.

The middle path is harder and slower. It requires building genuine technological capability while maintaining a clear view of the problem, the market, and when to make the shift toward being product-focused. It means building partnerships and finding market validation without a “product”. It means having one finger on the commercialisation trigger, while your foot is on the technology accelerator. That tension has been a constant across every stage of my career.

The cost is time, capital, and real risk. The payoff is not just a company, but a paradigm shift.
That tradeoff is the whole point of deep tech, and the companies that fail or stagnate in this space either do not understand it or lack the patience to honour it.

Endings and Beginnings

Summary

In this post, I’ll review at the last 2 years of VRTX Systems and talk briefly about the challenges/opportunities that led me to sidelining it.

VRTX Systems; the pitch.

So a little background; a little over two years ago I left a good job and a great team @ Arkeus to set out on my own. There were a few motivating factors; the desire to work for myself, and recognition of some significant gaps in the market when it came to building distributed robot systems. In particular, there was (and continues to be) significant hype around drone swarms and autonomous robot collaboration despite there being at least one significant engineering barrier that is yet to be overcome, at least in the open/civilian market – relative drone positioning & communication.

I was/am mostly excited about using distributed robot collaboration on heavy lift copters – for places where you want a crane, but can’t get one there – coastlines, shipping, mine sites, search-and-rescue and (in the defence setting) contested logisitics. Key feature being that the environment does not have reliable infrastructure, making having portable heavy lift a benefit. By doing this with drones, you might not be as cost-effective as a hiring a helicopter, but you’ll have a smaller per-unit cost, and if swarming technology is good enough, then the ability to scale to fit the problem. All of this assumes that each drone in the fleet knows the relative position of their neighbors.

GPS is by far the most common drone positioning tool, is passive and easily scalable, but vulnerable to spoofing/jamming, and only provides a location for the robot with the GPS receiver – you need a communication network as well. To get drone-to-drone position estimates, you would measure your GPS location, and broadcast that to the network.

Most of the current communication technologies are not designed for distributed operation – both WiFi & Cellular usually require a centralised node. Those that do (e.g. BATMAN) are set up to mimic the existing BSD network stack enforcing a layered approach, which makes less sense in a network of highly mobile communicators. In traditional networking, we decouple the network address from the physical location, for networks of mobile agents, we want the exact opposite.

VRTX Systems first product (VRTX RDIO) was indented to solve these problems by providing an encrypted mesh communication network with relative positioning for drones; meaning low power, hard-realtime, secure, resilient to interference and scalable. The key to solving this problems was to implement a secure decentralised network clock so that network nodes could measure time-of-arrival of packets to estimate distance.

Over the course of VRTX Systems, I got to the point where i had

a mostly working software stack on the embedded radio base on a STM32 LoRa development board & DecaWave unit),
broadcast-based communication over LoRa,
a distributed secure handshake & AES encryption scheme that could handle nodes joining and leaving,
Partial Network clock synchronisation.

I had hit a bit of a wall around November 2024, the key problem was that the network sync algorithm required either a coarse clock to separate the synchronisation into rounds (not scalable), or the development of a fully asynchronous version. The first case was an engineering problem, which would take time to solve (and probably a new board, since i was outgrowing the memory capacity of the STM32L0 chip). The second case required actual research; for me to take off my engineer hat, and put on my mathematician hat. The first cost time & money, the second cost time & risk.

The obvious decision was to implement the coarse clock sync, it was more important to move closer to a product than to “get it right”, but it was a little disheartening nonetheless. Fortunately, around this time I had a opportunity to work with my friends at Arkeus again, which gave me some breathing room, a team environment, and some new experiences.

Learnings

Embedded software is hard.

I knew this going in, but i keep getting reminded of it every time. Embedded software development is a different beast. In most application development, you expect that your vendored code is either a binary blob with header files, or source code at works, or a library provided by your OS or development envirnomnet at best.
This is a lot less standardisation in embedded development. That radio you’re using? Someone from ST has put together a demo application but it’s a demo – heaps of features listed on the device datasheet are missing and there is a lot of drudge-work to get the stuff you want implemented in the way that you want it.

There is also a whole lot of churn. I settled on the Arm M-bed 2.0 ecosystem initially since it seem to have the friendliest development experience, only to have Arm pull the plug on officially supporting it 2-3 months later.

The biggest win was getting familiar with embedded rust, particularly the RTIC and embassy-rs libraries. The next bare-metal project I work on will most likely use one of these. I definitely feel that with a few more tweaks embedded rust will be the best embedded development environment by far.

Don’t underestimate low TRL risk.

Technology Readiness Level (TRL) is a common way to measure how mature a particular technology is. Generally speaking, academic publications are between TRL 1 and TRL 3, while things you can buy on the shelf of your favourite tech store are TRL 9+.
Generally speaking, my personal “sweet spot” is to pick things at low TRL (say 2-3) – academic publications, prototypes etc. – an drag them up to TRL6-ish – i.e. a working system with almost no design/implementation risk left, but still heaps of risk around deployment & sustainment. This gap (TRL-4 to TRL-6) is the so-called “valley of death” because it is here where many technologies fail on their way to commercialisation.

There are many reasons why this happens, but the one I want to focus on is “assumption propagation”. When you bundle a bunch of technologies together making then co-interacting, the operating requirements of one technology puts operating constraints upon another. These interactions cascade out in a way that can be pretty unpredictable. Best case, you can handle it. Worst case, it just straight up won’t work and you need to go back to the drawing board.

The best way around this is to build the smallest, dumbest, dirtiest, “complete system” prototype, but to build it in a way that makes iteration, learning & discovery the objective.

My mistake with VRTX RDIO was that I focused on the engineering risk, and iterated on the embedded software, before appropriately de-risking the clock sync. At the end of the day though, I think it was a damned-if-you-do-damned-if-you-don’t, since there was a whole bunch of risks associated with the radio board development that I had to work through.

The solo founder experience is not for me.

I’m very familiar with working solo for prolonged periods of time – a PhD. in maths will do that – but the solo founder experience is a different beast. When you do a PhD., you focus deep on one topic with the help of a supervisor, the camaraderie of your fellow candidates, and ideally feedback from your school/wider community.

As a solo founder, you’re frequency context switching to solve whatever the most immediate problem is (software, firmware, electronics, design and business), with no established community (you have to take time out of work to do that) and no real feedback unless you’re open-sourcing (different business model) or you are at the point where you have customers/clients. This might work well for some people, but it doesn’t for me. I think it would be fine if i were not “on a timer” (i.e. when the allocated savings run out), but as it was I struggled to stay positive about getting VRTX sustainable quickly enough.

Doing some freelance work really highlighted for me how much being part of a team means to me, and how much more effectively I work when I know what I contribution is part of a larger cause.

The future of VRTX Systems

For now, VRTX Systems is on ice while I focus on other things.
Details are for another time, but I’ve founded a company – Pneumatica.Bio – with some friends/colleagues from my days in the Systems Biology Lab.
You’ll be hearing more about Pneumatica very soon as we have a pretty massive 2026 coming up.

Thanks for reading.

Iron Crosses in Software Engineering

Iron crosses describe the trade-off space of a solution to a problem, and occur due to competing demands on resources. The classic one is for project management: good, fast, cheap, done. It is often assumed that ‘done’ is non-negotiable, so that trade space just ‘good, fast, cheap; pick two’.

In software engineering, an important trade space is between simplicity, generality, efficiency, and punctuality; or more casually; simple, general, efficient, done.

Simple solutions are cheap (in terms of effort) to maintain, cheap to change, cheap to onboard new maintainers, and easy to hand over.
The fundamental constraint is information/understanding/context and how much is required to understand the solution.

General solutions solve a wider variety of problems, including the one in question, such that you only need one solution for a wide class of problems. Maintenance cost is be amortized over many projects, and duplication of effort is minimize.
The fundamental constraint is scope/abstraction and how many different features are expanded when implementing the solution.

Efficient solutions are cheaper (in terms of dollar cost, or energy) to deploy, more responsive and have lower hardware requirements.
The fundamental constraint here is hardware affinity and how tightly coupled the solution is to the particular set of hardware.

Punctual solutions are done on time and under budget.
The fundamental constraint here is resources; how many (and with experience) are allocated to the solution.

You want them all, but can only choose three.

These are competing constraints, so deliberate or otherwise, a choice is always made.
Better tools make the return on investment cheaper, but can’t make those tradeoffs disappear.

Both engineers and product managers have (usually implicit) assumptions/preferences about how these trade offs are to be made for a given problem. Preferences vary depending on background, culture and tech. Application / library developers tend to like general & efficient, systems programmers tend towards simple & efficient, web development towards towards simple & general. Product managers generally want simple and done.

Failures occur when stakeholder trade-off expectations don’t align.

Example: developer wants to implement a general and efficient solution since they’ll expect to need it later for a different project. Product manager wants it done quickly and simply due to timeline and resource pressure.

Possible outcome: Developer sees PM as making future pain, impacting morale and causing frustration. Product manager see developer as not aligned with business goals and having the wrong priorities. Thing gets shipped but nobody is happy about it.

Usual outcome; developer delivers it late, it’s over-complicated and hard to maintain. It’s also buggy and needs performance issues fixed before it’s usable. If it gets shipped, it gets shipped late and nobody is happy about it.

It is crucial to find alignment between stakeholders in order to reach a mutually satisfactory solution.

Beaglebone Black Baremetal

The Beaglebone Black and Baremetal programming.

The BeagleBone Black (BBB) is a system-on-a-module (SoM) based on the TI AM335x CPU series. It’s relatively cheap, extendable via ‘capes’ (expansion boards) and has some interesting features such a programmable real-time co-processor units (PRU). I bought this board to mess around with real-time operating systems (RTOS) and bare-metal programming. Maybe not the ideal stating platform, but there is plenty of room to grow.

Even to someone comfortable programming in C, the embedded space seems somewhat impenetrable and seems like a combination of antiquated tooling and black magic. My aim here is to fix some of that – which will require some experimentation, exploration and iteration.

Host machine setup.

I’ll be using Ubuntu 20.04 as my development environment, not by choice – I would have preferred to use windows subsystem for Linux (WSL) as I’m running on a PC but WSL doesn’t handle USB very well, and I’ll be needing that to talk to the BeagleBone.

Some things we need:

A BeagleBone black.
A USB to TTL serial converter cable (I bought this one, and it seems to work just fine).
A microSD card (and some way to flash it)
GNU cross-compiler toolchain (apt package gcc-arm-none-eabi) and newlib
GNU make and associated build tools.
TI Starterware with the BeagleBone Black patch. (https://www.ti.com/tool/STARTERWARE-SITARA)

I’ll also be using git for source control, but you could just as well use svn.

Issue The First: Installing TI Starterware.

My first step was to ensure that I could build and deploy the TI examples onto the device. Not surprisingly, I ran into issues almost immediately. Firstly, TI Startware package is distributed as a 32-bit ELF binary, which means it wont run out of the box on a modern Linux distribution. I found this out via a bit of digging on stack-overflow, which pointed me at the Linux application file – which spits out information about the file type, surprisingly enough. Further digging on stack-overflow suggested that i needed to run the following to enable the execution of 32-bit binaries;

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386(

(As an aside, one of my frustrations with Linux – and technology in general – is how much working out how to use the software feels like memorizing incantations. More on this another time…)

The i386 issues is a feature of just how old the library is. TI have since replaced this with a new processor SDK which we may end up exploring later. However, the new SDK is tied to TI’s CodeComposer Studio – an Eclipse variant IDE for the TI chips. Personally, I wish TI was a bit more tooling agnostic, and spent more time and resources on documentation instead of ‘helping’ via an IDE. I get the impression that this is standard practice in the embedded world, so I guess there are reasons for it?

TI provides a patch for the BeagleBone Black, which can simply be extracted into the directory where the Starterware package was installed.

Because I like to make things difficult for myself, I elected to install the latest version of the arm GCC cross compiler toolchain (x86_64 to arm-none-eabi) directly from the arm developer website and installed newlib (libnewlib-arm-none-eabi) via apt.

At this point, we’re ready to try building the demo applications.

Issue The Second: Building TI Demo Applications.

Navigating the Startware SDK was pretty confusing at first, and the documentation was not much help. Eventually, I found the entry point to the build process in build/armv7a/gcc/am335x/beaglebone/makefile. Unsurprisingly, it did not ‘just work’ and so begins the task of debugging the build process.

The first thing was an easy fix; the version of cross-compile toolchain needs to be correctly specified. To work out where this is set, we notice that the entry point makefile simply calls make in each of the specified sub-directories. Each makefile in a particular sub-directory then includes build/armv7a/gcc/makedefs which is where we want to look. There, on the first non-comment line, we find the toolchain definition.

build/armv7a/gcc/makedefs - line 42
# LIB_GCC=${LIB_PATH}/lib/gcc/arm-none-eabi/4.7.2/   <- Old toolchain.
LIB_GCC=${LIB_PATH}/lib/gcc/arm-none-eabi/10.3.1/
LIB_C=${LIB_PATH}/arm-none-eabi/lib/

The other thing to note is that the environment variable LIB_PATH needs to be set so as to point to our cross compile toochain. Since I’ve install this into /opt, I’m exporting LIB_PATH=/opt/gcc-arm-10.3-2021.07-x86_64-arm-none-eabi in .bashrc.

Now, we should be able to compile some of the applications. Running make in the beaglebone directory does build some applications, but our console is being inundated with warnings about -march and -mcpu being incompatible. With a bit of digging, it seems that this is a change in modern GCC (vs the old 4.7 version the SDK shipped with), and that specifying the architecture is redundant. To fix this, simply delete the flag -march=armv7a from the CFLAGS on line 167 of build/armv7a/gcc/makedefs.

The final issue I encountered was a redefinition of strnstr in the demo application. My hypothesis is that the version of GCC/newlib that this SDK was built against didn’t have strnstr implemented, which is required for the lightweight IP (LWIP) stack that the demo application uses. A bit of grep-ing discovered that LWIP provides a compiler define for this exact situation, meaning that the fix was an easy on – just add DLWIP_HTTPD_STRNSTR_PRIVATE=0 to the LWIPCFLAGS make variable in the demo application makefile (line 91 of build/armv7a/gcc/am335x/beaglebone/demo/makefile).

Issue The Third: Booting the Beaglebone from an SD Card.

The Starterware documentation says that any ‘FAT’ filesystem is sufficient for boot loader, and that one simply needs to rename the build artifacts with TI headers (boot_ti.bin, demo_ti.bin) to MLO and app respectively, for the bootloader to pick it up. Again, not so simple. The general approach here is as follows:

Format the microSD to some FAT filesystem (W95 FAT32 LBA seems to work just fine.)
Copy across the MLO and app binaries.
Plug in the USB-TTL cable and point your serial application (minicom for me) at the corresponding tty port (/dev/ttyUSB0 for me).
Insert the microSD card, and power cycle the beaglebone, holding the S2 switch (near the SD slot) on boot.

If this works, you should see a bunch of activity on the COM port.

I encountered two fail-states. Firstly, if the card is not flashed correctly, you may receive ‘CCC‘ from the serial port. I couldn’t find out what that means, but I’m sure it’s not great. It was mostly likely an issue with the filesystem, or the onboard bootloader couldn’t find the MLO binary.

In the second fail state, the COM port registered that the bootloader had started as ‘StarterWare AM335x Boot Loader' was sent down the COM port, but it failed to move onto the next stage. A bit of printf debugging isolated that the bootloader was getting stuck calling f_open. I have no idea why, but it does seem that this bug is not present in debug builds. I don’t have a JTAG emulator (which I think is what’s needed to debug this sort of thing) so I can’t really get too much insight into what’s going on. I guess it’s not too much of a concern for me though, since I’ll probably not be using the Starterware bootloader for too long. If worst comes to worst, the BeagleBone Black patch provided by TI comes with a pre-built bootloader, so I could always just use that, or (more likely) piggyback onto Uboot.

Next Steps

With the TI Starterware working, the next steps will be some tooling and planning. In particular:

Constantly copying across binaries to an microSD card is pretty inefficient. The existing bootloader (uboot) allows for booting over the network. Can we use that to make development easier.
The TI way of building doesn’t suit my tastes and the source is all over the place. Can we factor the ‘BeagleBone Black’-specific parts of the SDK (and bin the rest) into a something more suitable for what I want to explore.
What needs to happen to get the video working? Either via HDMI or LCD.

Til next time.

A foray into articulated robots.

The Darkpaw is a hobbyist quadruped kit that I’ve been playing around with for some time now. The goal is to learn how to model, control and program articulated robots for fun and (maybe one day) profit. This post will give a bit of an overview of how i got started on this, where I’m at, and lay some groundwork for future posts.

Into the web of legged robots.

Legged robots are pretty awesome (see this video by Zenta)

(As a aside – I showed my partner this video and now she wants one to keep her company on jogs so as to light her path and further intimidate the locals…)

or this one by Matt Denton…

Anyway, not long after the start of lockdowns during the COVID-19 pandemic in Melbourne, I found myself looking for a new project. I had been working as a post-doc at the Aero-Mech flight lab, and had been learning about underactuated robotics via Russ Tedrakes excellent course from MIT (very highly recommended). Additionally, I had been working mostly in Python (and ROS – ugh – but more on that another time) and developing/re-discovering my C chops. Upon this backdrop, and inspired by a random incursion into the youtube hexapod rabbit hole, I figured it’d would be fun to actually learn how to program such a robot.

Enter the Darkpaw

After a bit of rummaging around ye-olde-internets, I found the Adeept Darkpaw. It seemed to suit my needs; had most of the parts including motor driver hat, servos, camera and light – all that was needed was a Raspberry Pi 3b+ and some batteries. The ‘code’ was all python, as is the trend in hobbyist robotics, so it would be trivial to reverse-engineer to do whatever i wanted with it. So i got it and then got to work.

Two issues were immediately apparent. Firstly; neither my power adapter nor the two 18650 batteries were sufficient to keep the Pi from going under-voltage and shutting the thing down. (I’ve recently upgraded my power supply to a higher power, and i still get undervoltages, but at least the Pi doesn’t reset).

Secondly. Well. The shipped code was… lacking, and we’ll leave it at that.
Of course, this did not really bother me, (other than the general feeling of distaste when looking at it) since the plan was to re-write my own implementation.

The assembly was pretty straight-forward, and there were plenty of spares just-in-case. I wish it was easier to get the Pi in and out, or at least to access the SD card, but one can’t ask too much of such a cheap kit.

The biggest learning curve has been getting to grips with kinematics in practice. Given my background, one might expect me to find rigid body modelling ‘trivial’. The reality is that it took me a number of attempts to get it right – partly due to the fact that the system has algebraic constraints. This is a consequence of the design, as the motors do not drive the leg joints directly, they instead translate force through levers and links. I’ll save the maths for a later post.

Current State

A sweep motion running on all legs.

Currently Implemented:

Motor / LED drivers (via user-space i2c).
Forward and inverse kinematics.
A basic sweep/reset walk cycle for testing.

Next goal is a footstep planner!

About Me

Welcome to the personal site of Dr. Peter Cudmore. Please enjoy your stay.

If you are what you do, then I spend my time:

Working as a postdoctoral researcher in optimisation and control with a deep interest in complex systems and nonlinear dynamics as applied to engineering in the broadest sense.
Messing around with robotics and embedded computers.
Learning/thinking about philosophy, psychology, robotic/systems intelligence and cognitive science.
Playing guitar/bass/synth.

I have

Obtained a Ph. D. in applied mathematics, studying synchronisation.
Played music professionally.
Wrote software prototypes for distributed control of quadrotor swarms.
Worked on cross-domain modelling tools for biochemical-electrical-mechanical systems.
Designed and built an onboard preamp for my bass.
Worked on control and camera software for underwater remotely operated vessel.