The network cryptography design, implementation and hand-off to the development teams that has occupied much of the last two years for me is almost done. Feels good to be nearing the completion of this work. It has been a real challenge but I’m happy with the end product. Slowing down as the remaining items are more varied and less technically challenging…but still need to be wrapped up.
Not sure what comes next though it looks like there will be more clean-up after the main work gets approved and shipped. Hoping for an interesting slate of new challenges to dig into as we enter the new year.
Covid remains a challenge. Mostly working remotely as with omicron in play we’re back to at most one day in the office each week. Various complications on the home front are distracting but things keep moving forward and we’ll get this wrapped up with a bow on top soon enough.
I watched a few conference sessions talking about software process yesterday evening.
There was one discussion of agile that I really liked. I’ve seen quite a few parts of agile processes that I think add value but I’m not convinced that the overall processes map well to the sorts of large, embedded software projects that I’ve generally been involved with.
Another session made some very good points related to dealing with technical debt. The presenter had some very interesting thoughts on using source code control system information to direct refactoring efforts. It makes sense that modules with significant complexity and lots of ‘touches’ are good candidates for clean-up. He also made the excellent point that attacking issue counts will focus everyone’s attention on the small, low risk, low reward items that can run down the count quickly. Dealing with a thousand minor naming issues will do little to improve tangible code quality. Addressing a single, large snarl of complex interactions may result in huge improvements. I long ago realized that uncritical use of metrics can derail a team faster than anything else.
There was also a short functional programming session that seemed decent. It still didn’t address my biggest pain points with functional though.
He made a decent case for the merits and limitations of functional approaches. I tend to buy them for ‘business logic’ type work.
The place I run into trouble with functional (and to a less extent the JVM languages in general) relates to numerics work.
I’ve spent a chunk of time processing images and signals. I still don’t see how one can reasonably implement something like noise reduction or sharpening of a reasonable size image (say 5000 x 5000 RGB pixels) in a functional language. Immutable arrays would seem to leave the developer with a lose/lose/lose set of choices.
- Direct approach – Process each pixel in series and create a new image array with the updated pixel. Repeat this 5000×5000 time, copying the array at each step.
This obviously fails even in a garbage collected environment as the process of copying megabytes of data for each pixel update will kill you. It does mean that access to adjacent pixels should be reasonably fast as reads will be no different than in a procedural environment.
- Partitioned Tree approach – Build your large array as a tree structure under the covers with (say) 32 elements per leaf node. This was the approach suggested by the presenter in the talk.
This seems to be the worst of all possible worlds. Read access to each pixel requires traversing several levels of pointer indirection. If 32 pixels in a row are updated then we generate 31 ‘dead’ copies of the segment along the way. Locality of reference becomes a mess as adjacent items in the ‘array’ may be anywhere in memory.
- Process and Reassemble approach – Run through the entire source image and generate a change list to be applied. Once you’ve finished with the image, generate a new array with all of the specified changes in place.
This will potentially generate a list of tens of millions of update entries in memory. I’m also not sure how one implements the ‘create a new array based on this array with these updates’ in an immutable environment. I guess this is the bottom line of all of the functional coding I’ve run across…the functional environment seems to always assume that there is procedural ‘magic’ to make the ends meet. I suspect that the answer would involve writing the image processing primitives in a ‘real’ language and then exposing them as unitary operations on the functional side. This rather strongly suggests that functional languages will always be a specialty item and not the ‘main course’.
I’ve been spending some time looking into password hashing best practices over the last week.
I’ve know about the BCrypt algorithm for a long time as the old BSD standard ‘high effort’ hashing algorithm designed to make brute forcing hashes difficult.
I’ve found that there is a new effort called SCrypt intended to generate a modern equivalent for dedicated password hashing as well as a ‘password expansion’ algorithm that appears to be in wide used called PBKFD2.
The PBKDF2 algorithm applies an HMAC using the key input to inject the salt and then to chain iterations of the process. It takes a user selected number of iterations that allows the work-load to generate the hash to be tuned to the scope of expected attacks (and to the performance of the target hardware). This allows modern high performance algorithms such as SHA-256 to be applied in a manner that makes the total calculation of the final salted hash resource intensive enough to reduce the likelihood of a successful brute force attack.
I had a short conversation yesterday about securing UDP data. When I dug around little it became clear that there is an existing, RFC documented protocol for handling that. I haven’t yet read the specification (though I likely will as it is an interesting technology).
There is a wikipedia description here and the primary RFC is here.
Being able to secure unsequenced and unreliable datagram traffic using a design that is reasonably well vetted seems extremely useful. There are places where UDP is uniquely useful and security is becoming a much larger issue in the market today.
Interesting looking sample code here.
This weekend I’m roaming a bit afield of my normal haunts.
I’m looking at service hosted JVMs in windows and in particular having such a service interact with the SERVICE_CONTROL_PRESHUTDOWN message. Handling this message allows a service to defer shutdown while handling activities that it needs to complete before the actual shutdown processing begins. I suspect it may also allow that service to call AbortSystemShutdown() in order to cancel the shutdown.
I spent my sandbox time this weekend bouncing between JavaFX and various java spring topics.
I have pretty much concluded that JavaFX isn’t somewhere I want to spend sandbox time. It appears to be less capable and productive than WPF for local UI work and Angular on Spring appear more broadly useful in the environments where Java reigns supreme.
Continue reading Monday morning after some java spring for the weekend… →
Ok..and Kestrel seems to be the .NET Core alternative to the OWIN stuff under .NET classic.
Yet another option…in my case the OWIN side is likely more interesting as I’m almost certainly going to be doing things that need interop or similar windows-centric functionality. Interesting as a Linux facing option though. May also be lower overhead in cases where a particular micro-service doesn’t need access to native capabilities.
I am also expecting to need some sort of SSL certificate to enable TLS on these links (don’t need the full commercial cert probably as these are likely going to be expose by IP address and not on the open web). I need to understand what is needed to deploy TLS, ideally with cert verification on both ends using certs I’ve created myself and that don’t correspond to a particular URL.
In this case I’m looking to ensure no MITM attacks and to encrypt the traffic but not to ensure much more than that. I don’t want further authentication to leak and I want to protect the connection (for example for a web UI on a small ‘appliance’ that may at times be exposed to an open internet connection).
Mostly saving these to read/watch in more detail later. Options on top of options here…
I continue to bounce back and forth between self-hosted web UI and WPF UI for implementing simple user interactions.
WPF is likely to be easier to build the right sort of thing with but is a bit less flexible and more ‘backward looking’. Web interfaces require a self-hosting solution if they’re going to be stand-alone and cross more lines of language and environment.
On balance the web UI path is better as a learning experience and aside from a few corner-cases (OpenGL say) likely to result in a better user experience in the long run.
I’m still looking for the right self-hosting solution and UI framework solution. I’m going to have to settle on one shortly so that I can being experimenting in the ninecrows sandbox and getting a feel for the technology.
Current links of interest on the self-hosting front:
There also seem to be some hard choices between versions of the framework…classic and core in particular.
More to learn as things move forward. I expect I’ll wind up doing things with both/all before I’m done as they have different strengths. The sorts of things a WPF/WinForms/MFC UI can do in terms of digging into the system and being screen real-estate aware along make them of great interest.
I’m getting things going on some sandbox software projects that have been sitting on the shelf for some time. Perhaps a bit spread out but should be interesting.
I want to be able to put together free-standing service applications that expose both RESTful programmatic control and single page interactive GUIs without involving an external web server.
I’ve also signed up for several of the sale Udemy courses…started out with an impulse buy of the Unreal Engine Programming course and an associated Blender course. Added in some Angular, Node and web programming stuff. Probably more in there than I’ll have time to get to, but at $10 each I can deal.
Looking at some sandbox projects that need web interfaces.
Winding up with a big split here
- PHP/Apache (LAMP) for real site work.
Anything that needs to run up here on my hosting will be something LAMP-like. Code needs to be PHP (at least locally) with back end persistence in MySQL.
Front end should be more flexible. As long as the back ends can provide what the browser side code needs all should be good.
- IIS/ASP.NET looks like the easiest choice (with .net core MVC as the tool set) for windows integrated work. I have a few things that want to run locally on a system but present a web interface externally. These tools should provide lots of ‘traction’ with close windows integration. Probably an Angular front end and perhaps a full service on the back-end.
- Node.js shows up in many places. It provides a fast to deploy web server that is flexible and easy to code for. It is on my list in part because it has been showing up in places that are interesting and it part because it promises to support light-weight tasks where IIS would seem like over-kill.
For the front ends there are a huge number of choices out there. Of the ones I’ve seen
- Angular 2 seems to be pretty capable and well respected. I expect to do at least some work with Angular and if it proves easy to work with I may focus on it.
- Bootstrap is older (I think) but appears to provide a similar set of tools and infrastructure. If I run into issues with Angular, I may give this a try for something small and see how well it works.
I am also looking at free-standing, light weight web server options that run in C#. This is primarily aimed at RESTful interfaces to back-end code. OWIN and Katana seem to promise something like this while on the C++ side (and I expect C++ will show up now and then) microhttpd looks promising (with GPL restrictions, but these are sandbox projects).
Currently doing quite a bit of reading. Soon I’ve got to kick off from reading and get to some doing. Not sure what I’ll tackle first but I will probably try to list the things I’d like to kick around before I pick one…stay tuned.