Monday, Dec 10, 2018

Trying to get back on track with sandbox work at home.

I’m taking another run at the tool for archive management and de-duplication processing that has been on the plate for some time. This tool wants to read files, determine uniqueness of files (based on content) and maintain an index (likely in MongoDB) of files that have been scanned and are expected to exist. I’m writing things in C# mostly. The current user interface approach involves WPF but may also make use of ASP.NET (probably core) at some point. Looking to solve a problem and play with some coding tools on this front.

I’ve got better versions of my interop C++/CLI code to provide access to Win32 volume and file information functions in place and tested. I have not yet pushed these up to github as they’re still just free-standing code without anything to actively run them.

I’m expecting to put in place some more helper code shortly and then build out a simple WPF interface to run things. I also want/need to build a more capable MongoDB back end to hold file scan results as part of the process,

Holidays are a busy time…

Haven’t gotten much done on a technical front (aside from work items) in a while as other things have been dominating my time (partly work there as well).

Really going to have to get the RepRap work going once the workbench in the basement is cleared for real work (one of the areas that is taking time…cleaning the basement). Also been doing a quick emacs-lisp refresher as I’d like to get some things done on that front.

I did drag the intuos tablet down to the living room. I hope to find time to do some drawing and blender work here and there soon as that is less time consuming and can be done in small bits. I need to locate my corel painter install files and get that package on the living-room NUC. I suspect that responsiveness may be better there than on the big machine in my office too.

VirtualBox supports multiple monitors…

I’ve been using VirtualBox pretty heavily to partition things out on my systems where some tasks require setups that clash at the OS level.

I generally find multiple monitors to be exceptionally helpful in software development settings. I’ve been running with my Virtual Box VMs on one monitor and using the other monitors for host OS based windows…and this has worked pretty well.

This morning I decided to take a look at the configuration settings for my main VM and to my delight I found that it is possible to configure additional monitors and place them on the desktop. This should streamline things when developing on a VM as it allows me to keep my IDE full screen on one screen while using a second for other activities.

Now if I can only figure out how to make Docker and Virtual Box live together in harmony at home (Docker wants Hyper-V while Virtual Box can’t run under that hypervisor) I’ll be feeling pretty happy.

JSON over Stdin/Stdout for small local components.

I’m looking at adding a local control capability to my little C++ based file and volume identifier tool. I’ve run into a number of small functions that would be well served by tools written in specific languages that have access to specialized APIs but would benefit from a local interface with somewhat decent performance.

Unix style text in/text out is limiting as the command/response format lacks structure and the performance (with one process start per request) is potentially unacceptable.

Adding in a command line switch that sets the tool into ‘json in/json out’ mode and keeps the program alive until the pipes drop or an exit command is received should address these issues.

Ideally, the stdin and stdout would transport UTF-8 data containing complete JSON entities…one as a request and one containing the complete response. This can continue until the hosting program sends an EOF or drops the pipes.

I’m wrapping together a very lightweight JSON generator/parser to support this effort and will be following that up with some testing of C++/Java/C# (and perhaps others) sub-process and pipe management coding. If all goes well, I should have a nice little approach for wrapping small pieces of code that need to run in a particular language but must be hosted elsewhere with decent performance and a robust communications format.

Native extensions to Python

I spent a little time last night looking at the coding requirements for creating native (C essentially) extensions to Python. This has come up previously in conversations as another way to make native code accessible functions available for scripting by folks who aren’t development engineers.

The facility appears similar in style to that provided for writing perl extensions or JNI. No surprise here. I’ll probably try playing with this a little bit sometime soon just to get a feel for the full series of steps. The process involved in getting the *.dll or *.so file into the right spot to be usable seems a little bit opaque, but perhaps it is just so automated that it isn’t much of an issue.

I’m not a huge fan of Python but it has a huge following and seems to be a useful intermediate step between shell scripting sorts of activity and full powered implementation languages (C++/C#/Java).

I wish python had not chosen such a poorly conserved item as white-space to represent block scoping (the tabs/spaces issues are not generally fixable without making the representation issues worse). They also seem to have created a permanent 2.x/3.x schism where quite a few years after the major version upgrade there is still a substantial amount of code on either side of the divide. Perl 5/6 had a similar issue, but perl 6 never seems to have taken off so the impact is much smaller.

Some links that seemed useful:

JavaFX Event Handling

I’m finding myself needing a better understanding of JavaFX UI event handling and my basic JavaFX books seem to ignore most of that aspect of the system. It looks as if the wiring of UI events is buried deep enough and the handling is automated enough that most of the time there’s no need to directly interact with events.

I need to perform an action when a given UI panel is exposed after being hidden…a simple enough activity in most cases. I expect I’ll find the right controls soon enough, just a bit surprised that they’re not talked about as directly as I’m used to.

Looking as if the easy route to this may be to name the top level pane in a given area and then attach a ‘show’ handler to that pane. Hoping this works as this would make it pretty easy to do what I need to get done.

Some links that I’m recording for later convenience (some just interesting here not directly relevant):

The charting and 3D API items are of interest, but not part of this, but I’m trying to keep notes on where things were found so that I can go back and dig deeper later…a small amount of clutter for convenience…

New Version of the C++ Templates Book

There is a new edition of C++ Templates: The Complete Guide (2nd Edition) out. This has actually been in print for around a year, but I’ve been a bit focused on C#, Java and Javascript and not paying all that much attention to C++ developments for the last couple of years.

I’ve created plenty of template classes and functions in C++ but have not done much more than a bit of reading on the powerful template metaprogramming capabilities provided by the C++ template facility. With C++ 2011/2014/2017 and its language and runtime library enhancements, these sorts of approaches have  become more powerful and less complex.

I expect to buy a copy of this and read it through sometime soon as a complement to The C++ Programming Language, 4th EditionThe C++ Standard Library: A Tutorial and Reference (2nd Edition) and Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14 1st Edition. I’m still considering dropping money on a copy of the standard itself (ISO/IEC 14882:2017 Programming languages — C++, main page here) as the comments on the final draft that is freely available seem to suggest it is more of a mess than is usual. Interesting…the ANSI copy is at $116.00 which is significantly less expensive than I had thought.

So many languages…

…and tools and environments that matter. Back on C++ for the moment as the APIs I’m looking to use are only available to native Win32 code. I also want to broaden my comfort zone in C++ 2011/2014/2017 a bit more as I’ve been doing far more C#, Java and JavaScript lately than C++.

I expect to finish the current piece of tool code in C++ and then look at wrapping it (or perhaps a C++/CLI equivalent) in a WPF/C# UI tool for managing file duplication/archiving. Depending on how things go I might play with some ASP.NET Core/Angular code on this front too but not sure at the moment. I need to get a better handle on my current sprawl of archival storage without losing anything that matters.

Reading through Effective Java Third Edition and re-reading Effective Modern C++ (with some digressions into The C++ Standard Library Second Edition). 

It does take a bit of reorienting when switching between Java/C#/JavaScript and C++ as C++ is the only language where most things aren’t pointers (references, but effectively the same thing).  Keeping things local and cleanly life-cycle managed is a rather different mind-set than the more free-wheeling approaches that the garbage collected languages encourage.

JSON over stdin/stdout

I’m looking at adding a json over standard I/O pipes option to my tool from last weekend. I’m thinking of this as an intermediate option between interop type solutions (jni, pinvoke and such) and simple command line execution.

Adding in a –jsonrpc option to the tool that selects a mode where json object are read from stdin and json results are emitted to stdout seems like a pretty good option. This should provide an abstracted but tightly coupled interface between a piece of code running as a separate process and a language that may not have access to the APIs needed to get the job done.

This does imply that there will be one subprocess per parent process that needs access. The upside of this is that things remain modular where a problem with one subprocess has no effect on others in the system. We sacrifice a bit of efficiency along the way, but I can’t see that as a major issue in most cases.