top of page
  • Writer's pictureDinesh Thogulua

System partitioning: When to implement a functionality in hardware

This used to be a question only relevant to system architects working for large firms: For ex., as a audio hardware architect in NVIDIA, every time we needed to implement a functionality, say, mp3 decoding or audio mixing, we had to ask ourselves if it makes sense to implement it in software or in hardware. However FPGAs are becoming ever cheaper and even startups now can have custom VLSI blocks in their product. In this blog, I want to convey my thoughts on how to decide which cases merit a hardware implementation and which cases don't.

When in doubt, use software

When you don't know if customer will see value in a product/feature, implement it in software. Don't worry if it isn't the most effective or efficient way of implementing it. The emphasis should be about finding out if the customer really cares - if your product/feature solves his problem or not. With hardware, even if it is done in FPGA, it takes way more time implementing, testing a functionality than software. You don't want to waste 1 minute of your time on an unproven assumption about the customer.

Even applications where FPGA or a IC stand out compared to software, namely high bandwidth, low latency applications, it is OK to implement a low bandwidth, high latency proof-of-concept in SW. Take some ML application for ex: Segregating good apples from bad apples (literally) using computer vision, ML. The actual usage model may be that you are scanning 50 apples every second as the fly from one conveyor system to the next, and instruct a set of metal sticks which apples to knock out (This is actually how it is done today :) - Watch it on youtube, it is fascinating). Eventually, to make decisions that quickly on that sort of volume may mean that you have to implement your computer vision, ML algorithms in FPGA with very high parallel processing. But for a prototype, you can implement in software and figure out a way to demonstrate its successful working to the customer - And then convince the customer that you can speed it up eventually. The point is to figure out if your customer even cares - It could be that the labour costs are so low that he/she decides to continue using manual labour for the task. It may also be that the way the apples are grown, they may naturally be 95% good and it doesn't make sense to have a product just to remove the 5%. Or the customer may have some other reason you hadn't even imagined - That is the point: You never know. So don't waste your time in HW without evaluating your assumptions about customer.

Standardised functionality = Hardware

If you are developing a functionality that adheres to a standard like ITU, MIPI etc., there is no question of customer validation - You either accept that customers want it (say 5G or SoundWire etc.) or you don't. For such functionalities, it makes sense to go to hardware directly - In fact, in such cases, it is usually a race to a product that works at the expected speed/latency, power consumption etc. In other words, optimisation is the name of the game and that means hardware.

I think FPGAs really shine here because you don't want to go directly to a silicon (ASIC or a block in an SoC) at the early stages of a standard. In fact, having represented my employers or participated behind the scenes for ITU/IEEE standards, it is the case that companies start working on implementing a technology even before the first draft of the standard comes out. You can then see each member company fight with others to push the standard in the direction that suits their implementations the most. Anyway, the point is, if you start developing at early stages of a standard, then you know that you have to keep incorporating changes as the standard evolves. This is why FPGA makes more sense than a silicon implementation.

Some corner cases

What if the functionality is something not necessarily related to a standard, but nevertheless well defined? For example, Fast Fourier Transform (FFT) is well defined. Doesn't it make sense to then implement it as a hardware accelerator? The answer is yes if there is a high bandwidth/low latency requirement on that functionality and the CPU scheduling is unpredictable like in a general purpose OS. In other words, if the CPU is running on RTOS and the set of tasks it needs to do are fixed and predictable, try your best to avoid going to hardware. The reason is, customer's penchant for higher performance would mean that you need to keep increasing the resolution of computations in your functionality. Clock speeds are ever increasing and that means a SW implementation

Time to market is the name of the game today for any product. Any USP a product has disappears in a few months as the competition catches up. Unless the USP itself is some optimisation (lowest power consumption, fastest rendering in the market etc.), it makes sense to implement functionalities in software that use them. Software is simply more aligned to today's fast evolving markets.

7 views0 comments

Recent Posts

See All



bottom of page