Category Archives: Software Process

Software Process

I watched a few conference sessions talking about software process yesterday evening.

Agile

There was one discussion of agile that I really liked. I’ve seen quite a few parts of agile processes that I think add value but I’m not convinced that the overall processes map well to the sorts of large, embedded software projects that I’ve generally been involved with.

Technical Debt

Another session made some very good points related to dealing with technical debt. The presenter had some very interesting thoughts on using source code control system information to direct refactoring efforts. It makes sense that modules with significant complexity and lots of ‘touches’ are good candidates for clean-up. He also made the excellent point that attacking issue counts will focus everyone’s attention on the small, low risk, low reward items that can run down the count quickly. Dealing with a thousand minor naming issues will do little to improve tangible code quality. Addressing a single, large snarl of complex interactions may result in huge improvements. I long ago realized that uncritical use of metrics can derail a team faster than anything else.

Functional Programming

There was also a short functional programming session that seemed decent. It still didn’t address my biggest pain points with functional though.

He made a decent case for the merits and limitations of functional approaches. I tend to buy them for ‘business logic’ type work.

The place I run into trouble with functional (and to a less extent the JVM languages in general) relates to numerics work.

I’ve spent a chunk of time processing images and signals. I still don’t see how one can reasonably implement something like noise reduction or sharpening of a reasonable size image (say 5000 x 5000 RGB pixels) in a functional language. Immutable arrays would seem to leave the developer with a lose/lose/lose set of choices.

  • Direct approach – Process each pixel in series and create a new image array with the updated pixel. Repeat this 5000×5000 time, copying the array at each step.
    This obviously fails even in a garbage collected environment as the process of copying megabytes of data for each pixel update will kill you. It does mean that access to adjacent pixels should be reasonably fast as reads will be no different than in a procedural environment.
  • Partitioned Tree approach – Build your large array as a tree structure under the covers with (say) 32 elements per leaf node. This was the approach suggested by the presenter in the talk.
    This seems to be the worst of all possible worlds. Read access to each pixel requires traversing several levels of pointer indirection. If 32 pixels in a row are updated then we generate 31 ‘dead’ copies of the segment along the way. Locality of reference becomes a mess as adjacent items in the ‘array’ may be anywhere in memory.
  • Process and Reassemble approach – Run through the entire source image and generate a change list to be applied. Once you’ve finished with the image, generate a new array with all of the specified changes in place.
    This will potentially generate a list of tens of millions of update entries in memory. I’m also not sure how one implements the ‘create a new array based on this array with these updates’ in an immutable environment. I guess this is the bottom line of all of the functional coding I’ve run across…the functional environment seems to always assume that there is procedural ‘magic’ to make the ends meet. I suspect that the answer would involve writing the image processing primitives in a ‘real’ language and then exposing them as unitary operations on the functional side. This rather strongly suggests that functional languages will always be a specialty item and not the ‘main course’.