Category Archives: Topics

Posts on technical topics. The sub-category provides the specific area of interest.

Software Process

I watched a few conference sessions talking about software process yesterday evening.

Agile

There was one discussion of agile that I really liked. I’ve seen quite a few parts of agile processes that I think add value but I’m not convinced that the overall processes map well to the sorts of large, embedded software projects that I’ve generally been involved with.

Technical Debt

Another session made some very good points related to dealing with technical debt. The presenter had some very interesting thoughts on using source code control system information to direct refactoring efforts. It makes sense that modules with significant complexity and lots of ‘touches’ are good candidates for clean-up. He also made the excellent point that attacking issue counts will focus everyone’s attention on the small, low risk, low reward items that can run down the count quickly. Dealing with a thousand minor naming issues will do little to improve tangible code quality. Addressing a single, large snarl of complex interactions may result in huge improvements. I long ago realized that uncritical use of metrics can derail a team faster than anything else.

Functional Programming

There was also a short functional programming session that seemed decent. It still didn’t address my biggest pain points with functional though.

He made a decent case for the merits and limitations of functional approaches. I tend to buy them for ‘business logic’ type work.

The place I run into trouble with functional (and to a less extent the JVM languages in general) relates to numerics work.

I’ve spent a chunk of time processing images and signals. I still don’t see how one can reasonably implement something like noise reduction or sharpening of a reasonable size image (say 5000 x 5000 RGB pixels) in a functional language. Immutable arrays would seem to leave the developer with a lose/lose/lose set of choices.

  • Direct approach – Process each pixel in series and create a new image array with the updated pixel. Repeat this 5000×5000 time, copying the array at each step.
    This obviously fails even in a garbage collected environment as the process of copying megabytes of data for each pixel update will kill you. It does mean that access to adjacent pixels should be reasonably fast as reads will be no different than in a procedural environment.
  • Partitioned Tree approach – Build your large array as a tree structure under the covers with (say) 32 elements per leaf node. This was the approach suggested by the presenter in the talk.
    This seems to be the worst of all possible worlds. Read access to each pixel requires traversing several levels of pointer indirection. If 32 pixels in a row are updated then we generate 31 ‘dead’ copies of the segment along the way. Locality of reference becomes a mess as adjacent items in the ‘array’ may be anywhere in memory.
  • Process and Reassemble approach – Run through the entire source image and generate a change list to be applied. Once you’ve finished with the image, generate a new array with all of the specified changes in place.
    This will potentially generate a list of tens of millions of update entries in memory. I’m also not sure how one implements the ‘create a new array based on this array with these updates’ in an immutable environment. I guess this is the bottom line of all of the functional coding I’ve run across…the functional environment seems to always assume that there is procedural ‘magic’ to make the ends meet. I suspect that the answer would involve writing the image processing primitives in a ‘real’ language and then exposing them as unitary operations on the functional side. This rather strongly suggests that functional languages will always be a specialty item and not the ‘main course’.

Fun with Password Hashing

I’ve been spending some time looking into password hashing best practices over the last week.

I’ve know about the BCrypt algorithm for a long time as the old BSD standard ‘high effort’ hashing algorithm designed to make brute forcing hashes difficult.

I’ve found that there is a new effort called SCrypt intended to generate a modern equivalent for dedicated password hashing as well as a ‘password expansion’ algorithm that appears to be in wide used called PBKFD2.

The PBKDF2 algorithm applies an HMAC using the key input to inject the salt and then to chain iterations of the process. It takes a user selected number of iterations that allows the work-load to generate the hash to be tuned to the scope of expected attacks (and to the performance of the target hardware). This allows modern high performance algorithms such as SHA-256 to be applied in a manner that makes the total calculation of the final salted hash resource intensive enough to reduce the likelihood of a successful brute force attack.

C++ 11, 14, 17 and Later

I’m quite familiar with much of the content of C++ 2011 as it represented a welcome and long desired step up in C++ language capability.

I’m less clear on the changes that live in the 2014 and 2017 incremental updates (smaller and more tightly focused) and the upcoming work that will feed into the next release.

Getting on top of this is becoming more important as I’m back in the C++ world and while almost everything should support C++ 11, the later iterations may be missing or fragmentary.

I’m spending a little time this afternoon looking through resources on this front, starting with the C++ 2011, 2014 and 2017 pages on wikipedia.

I have pulled the draft PDF file for 2011, 2014 and 2017 and grabbed the github source for the standard(s). These are quite useful, but seriously deep waters if only the changes are of specific interest. Interesting pointers on where to buy the official docs here with the 2017 version from ANSI at just over $100.00. The current working draft appears to be on github here. I may take a shot at building that into a readable PDF at some point…

I am also rather interested to see what is in boost these days. Back in the visual studio 2010 era, the TR1 content that eventually fed into C++ 11 was one of the bigger draws…now that is part of the core tools in general so I’m expecting a new range on interesting bits. The seem to have a github repo here.

And here is the C++ 20 page. Interesting that 20 looks to be much more like 11 than 14 and 17 (which were small tweaks).

Doing Some Unity Refresher Reading

On the flight to San Diego I did some Unity refresher reading and some thinking about game ideas that might be worth playing with.

I was looking for references to ScriptableObjects in the books I’ve got and didn’t find them. Guessing the focus there is too new to show up. I’ll post a bit on the ideas front over on PandaMallet in a bit.

It has been a slow summer on the home technical front and I really want to get that stuff rolling again as we move into fall. Plenty of interesting stuff to do, just need to find the time and decide to focus.

DTLS – Security for UDP

I had a short conversation yesterday about securing UDP data. When I dug around little it became clear that there is an existing, RFC documented protocol for handling that. I haven’t yet read the specification (though I likely will as it is an interesting technology).

There is a wikipedia description here and the primary RFC is here.

Being able to secure unsequenced and unreliable datagram traffic using a design that is reasonably well vetted seems extremely useful. There are places where UDP is uniquely useful and security is becoming a much larger issue in the market today.

Interesting looking sample code here.

Some Thoughts on Agile

I’ve done some development in an agile/scrum environment. There are a number of things it brings to the table that I see improving code quality in some ways and providing a more stable delivery schedule.

With all that being said, I don’t believe it is a silver bullet and I become a bit annoyed when I read books and articles that present it that way. I’ve been doing some refresher reading lately as we’re working with customers who run agile teams here and I’d like to help our team adopt useful bits of agile without harming our overall effectiveness. This is particularly challenging in a regulated environment like the medical device development we do here. It is also challenging when we’re doing contract development engineering and customers expect to have a contract that covers the work we’ll do before they start paying.

Stand-up

I’ve been running some sort of daily stand-up long before I heard of agile or scrum. If anything, scrum environments seem to make stand-up longer and more formal. In lead roles before I hit scrum environments, my stand-up usually involved going to wherever the bulk of the team was located (if I wasn’t already there…jobs varied) and having a short discussion with members of the team about how things were going.

One aspect of stand-up that I don’t endorse is the ‘blockers’ question that usually seems to be a rote part of the process. In a team of under ten people, there should never be blocking issues that last for more than a very short time. If you know who can help you then just ask (by email if they’re not immediately present). If you don’t know who to ask then either ask your lead or ask someone else and follow it up from there. If someone consistently fails to help those who need assistance then the team needs to stage an intervention and make it clear that we work as a team.

Blocking issues should never persist for long enough to make it to stand-up.

I also tend to use stand-up as a platform to address team wide issues and support issues that people encounter. Another side-effect of the ‘keep it short’ philosophy of scrum stand-ups are ‘information free’ comments. Telling the team that you’ve closed issues/stories ‘1123 and 1127’ and are starting work on ‘1134’ doesn’t really help to share information with the team. It may help the leadership track progress, but they have plenty of tools to do that already if you’re using any sort of software to manage work-flow.

I think this is enough for one day…I’ll add comments in another page in the near future…

< Prev | Next >

A Weekend of Database and CertificateS

Spent some time over the weekend doing some more work on the MySQL database layout for the cluster game and working on getting self-signed certificates prepared for my various development machines.

PHP and MySQL

The database work went smoothly. Still largely on the whiteboard at the moment. I’ve also been going through a PHP re-familiarization as I’ll need to code this stuff in PHP for my hosting and I haven’t worked in that environment in some time. I did grab an evaluation license for PHPStorm a few weeks back, but I fear that was premature as I haven’t reached the point where I need such tools on this sandbox project yet.

Certificates

I finally took the time to create SSH certificates to permit direct logins to my linux machines from my windows systems. That part I’ve done many times before and it went flawlessly.

I created and installed self-signed certificates for various local systems and set up TLS on their Apache servers. The creation and installation went smoothly, but the end-result was not what I was hoping for.

After installing the certificates in several different ways on the systems/browsers involved, I still did not see the secure icon in the address bar. I’m not sure whether this is caused by the certificates being self-signed (shouldn’t be as I installed the keys directly from files into the trust stores) of something else I’m not doing properly. I’ll need to keep looking at that one.

I do want to verify that the connections are using TLS. If they’re encrypted but not ‘safe’ because they’re not signed by a major cert vendor then I’m probably ok with that. If the TLS handshake failed because they don’t have the right certs then there’s a bigger problem.

Sunday evening I started down the road to building a local CA to sign all of my certificates with. I’m wondering if setting this up and loading its public key as a trusted root may give better results. The process is a bit more involved but may be worth it if it gets closer to the results I’d get with a commercial certificate.

I still haven’t found a way to load a FreeTLS certificate on my GoDaddy hosting. One of these days I’ll spend the time to get on the phone with their support folks and see if this can be worked out.

I may try setting up a FreeTLS cert on my dynamic DNS connection that targets a port on my home firewall. That would provide more flexibility, but be less robust and scalable.

Samba SMB Shares

Toward the end of the evening I ran through samba installs on several systems. I had been pushing files around between my windows and Linux machines all weekend and wanted to make things closer to seamless.

I had no real luck on that front. I could get things to the point where windows recognized share names from the Linux machines. I could never get things to the point where my windows systems could connect to a share and see files inside. Not sure what I’m missing and the samba logs were not at all helpful.

I’ll probably re-visit this again sometime soon, but for now the convenience of having it working isn’t worth the effort involved in finding out why it isn’t.

Getting Back to PHP Work for Cluster

Since vacation I’ve been pretty busy working on photo post processing and around the yard.

At this point I’m going to be trying to get back to building a back-end for the unity based cluster game that runs in PHP on my web hosting (initial work on local sandbox PHP instances of course).

I think I’m going to try using Visual Studio Code with PHP Extensions to get this started. I’ve done a little PHP coding in the past, but this looks likely to be far more involved than any of that.

I’d like to get a TLS cert on my site before going live with this, but it appears that my hosting may not support free TLS or similar cert installs and I’m not happy adding the annual renewal cost for a cert to my site at this point so stay tuned. I’ll probably try hitting GoDaddy support some evening soon to see what they can tell me.

First steps will be getting a simple RESTful interface defined and then laying out some simple SQL schema to provide the back-end. If I can get that working, I’ll look at extensions necessary to provide the full back-end to the game as a whole. Not looking for commercial quality here, just something sufficient to allow multi-player turn based gaming.

CLuster Game Web Back Ends

Just reinstalled clean and up to date copies of XAMPP, MariaDB and MongoDB on one of my home machines. 

I need to write some RESTful PHP code that can run on my web hosting as the back-end for a VR game I’m playing with. This gives me a platform for building that code in a safe place.

I do need to stick to PHP 5.x features as my GoDaddy shared hosting does not support PHP 7.

I’ve got to get the basics stitched together, add an appropriate .htaccess to keep passwords out of inappropriate hands and then start working out a SQL schema that works for the persistent game data I need to store.

Spending a big chunk of this long weekend post processing pictures from our recent vacation (see them on the blog side of ninecrows if you’re interested). Watched the whole last season of Game of Thrones and saw Aladdin. Lots to still get done, but progress is progress 🙂