Native extensions to Python

I spent a little time last night looking at the coding requirements for creating native (C essentially) extensions to Python. This has come up previously in conversations as another way to make native code accessible functions available for scripting by folks who aren’t development engineers.

The facility appears similar in style to that provided for writing perl extensions or JNI. No surprise here. I’ll probably try playing with this a little bit sometime soon just to get a feel for the full series of steps. The process involved in getting the *.dll or *.so file into the right spot to be usable seems a little bit opaque, but perhaps it is just so automated that it isn’t much of an issue.

I’m not a huge fan of Python but it has a huge following and seems to be a useful intermediate step between shell scripting sorts of activity and full powered implementation languages (C++/C#/Java).

I wish python had not chosen such a poorly conserved item as white-space to represent block scoping (the tabs/spaces issues are not generally fixable without making the representation issues worse). They also seem to have created a permanent 2.x/3.x schism where quite a few years after the major version upgrade there is still a substantial amount of code on either side of the divide. Perl 5/6 had a similar issue, but perl 6 never seems to have taken off so the impact is much smaller.

Some links that seemed useful:

JavaFX Event Handling

I’m finding myself needing a better understanding of JavaFX UI event handling and my basic JavaFX books seem to ignore most of that aspect of the system. It looks as if the wiring of UI events is buried deep enough and the handling is automated enough that most of the time there’s no need to directly interact with events.

I need to perform an action when a given UI panel is exposed after being hidden…a simple enough activity in most cases. I expect I’ll find the right controls soon enough, just a bit surprised that they’re not talked about as directly as I’m used to.

Looking as if the easy route to this may be to name the top level pane in a given area and then attach a ‘show’ handler to that pane. Hoping this works as this would make it pretty easy to do what I need to get done.

Some links that I’m recording for later convenience (some just interesting here not directly relevant):

The charting and 3D API items are of interest, but not part of this, but I’m trying to keep notes on where things were found so that I can go back and dig deeper later…a small amount of clutter for convenience…

New Version of the C++ Templates Book

There is a new edition of C++ Templates: The Complete Guide (2nd Edition) out. This has actually been in print for around a year, but I’ve been a bit focused on C#, Java and Javascript and not paying all that much attention to C++ developments for the last couple of years.

I’ve created plenty of template classes and functions in C++ but have not done much more than a bit of reading on the powerful template metaprogramming capabilities provided by the C++ template facility. With C++ 2011/2014/2017 and its language and runtime library enhancements, these sorts of approaches have  become more powerful and less complex.

I expect to buy a copy of this and read it through sometime soon as a complement to The C++ Programming Language, 4th EditionThe C++ Standard Library: A Tutorial and Reference (2nd Edition) and Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14 1st Edition. I’m still considering dropping money on a copy of the standard itself (ISO/IEC 14882:2017 Programming languages — C++, main page here) as the comments on the final draft that is freely available seem to suggest it is more of a mess than is usual. Interesting…the ANSI copy is at $116.00 which is significantly less expensive than I had thought.

So many languages…

…and tools and environments that matter. Back on C++ for the moment as the APIs I’m looking to use are only available to native Win32 code. I also want to broaden my comfort zone in C++ 2011/2014/2017 a bit more as I’ve been doing far more C#, Java and JavaScript lately than C++.

I expect to finish the current piece of tool code in C++ and then look at wrapping it (or perhaps a C++/CLI equivalent) in a WPF/C# UI tool for managing file duplication/archiving. Depending on how things go I might play with some ASP.NET Core/Angular code on this front too but not sure at the moment. I need to get a better handle on my current sprawl of archival storage without losing anything that matters.

Reading through Effective Java Third Edition and re-reading Effective Modern C++ (with some digressions into The C++ Standard Library Second Edition). 

It does take a bit of reorienting when switching between Java/C#/JavaScript and C++ as C++ is the only language where most things aren’t pointers (references, but effectively the same thing).  Keeping things local and cleanly life-cycle managed is a rather different mind-set than the more free-wheeling approaches that the garbage collected languages encourage.

JSON over stdin/stdout

I’m looking at adding a json over standard I/O pipes option to my tool from last weekend. I’m thinking of this as an intermediate option between interop type solutions (jni, pinvoke and such) and simple command line execution.

Adding in a –jsonrpc option to the tool that selects a mode where json object are read from stdin and json results are emitted to stdout seems like a pretty good option. This should provide an abstracted but tightly coupled interface between a piece of code running as a separate process and a language that may not have access to the APIs needed to get the job done.

This does imply that there will be one subprocess per parent process that needs access. The upside of this is that things remain modular where a problem with one subprocess has no effect on others in the system. We sacrifice a bit of efficiency along the way, but I can’t see that as a major issue in most cases.

Adding Boost to the mix…

I’ve built and used boost before and had good results. I’m going to add boost to my home C++ tools mix and I suspect I may find a workable json parser/generator in there. Grabbing the pieces now, but I don’t expect that I’ll have a usable build for a day or two given available cycles/time.

Built b2 and now I’ve bootstrapped the zip archive of boost. We’ll see what is there in the morning after the .\b2 run finished. I think I may wind up missing some interesting bits that require optional extras, but I should have a decent subset and can likely add in the missing pieces as needed.

A bit of C++ over the weekend

I spent a good bit of time over the weekend digging into some java coding for work. Wrapping some things up and the extra time will help.

I noticed that there is a third edition of effective java out there. I have the older edition, but it is really too old to be helpful with the current state of the java world. The ‘effective’ books overall have tended to be very helpful with usage, idioms and anti-patterns beyond the basic so I’m looking forward to reading this volume and seeing what it has to offer.

I spent a good piece of today writing up a free-standing command line tool to display the unique file id for windows files. The API that provides this is exceptionally useful when working with file cleanup in systems where there are mount points, symbolic links and other reparse points present. This API is not directly accessible from languages such as java or C# and thus an invokable command line tool seems handy. Interop and jni would also offer access, but the command line tool seems the most general purpose (if perhaps low on the performance scale).

I wound up writing a small command line option parser to support this work (though only used a little at this point). I’ll likely keep this around and use it in other tools going forward.

I’m also looking at adding a json output generation option and a stream in/stream out switch to make this little tool more useful for higher volume solutions with programmatic interfaces. It seems as if a json based programmatic interface using standard I/O streams would offer significant performance while also providing enough ‘traction’ to make program to program use of the tool practical.

I ran into a small hiccup this evening as I was unable to upload the code to github as github appeared to be suffering from some sort of relatively serious failure. Hoping that tomorrow morning things will be in better shape and I can push the code up then.

I will probably build a simple json library for local use over the next couple of days (I’ve done this before and it is relatively straightforward as json is a very regular and simple format). I need small and easy more than capable and fast so something coded locally seems best.

Recent E-Book Purchases

Packt publishing, Manning and APress have been key sources of software related books for a variety of topics. Lately they’ve also been pushing out daily deals on eBooks that have grabbed my attention in a number of cases. Packt in particular has deep discounts on selected books (and often even deeper discounts if you buy all four on a given day) so they’re over represented in my list.

I tend to buy on a wide range of software topics as my preferred roles touch on all aspects of the engineering process and having some background across that range improves my effectiveness immensely. I’ve listed my purchase prices to give an idea of how deep these discounts go.

  • Game Programming using Qt 5 Beginner’s Guide – Second Edition ($8.00) Aug
    QT keeps coming up in discussions of C++ based UI programming. It is cross platform and seems to be well supported while the Microsoft specific alternatives all have limitations and most are seriously showing their age.
  • Hands-On GUI Programming with C++ and Qt5 ($7.20) Aug
    Another QT book covering core QT programming (the games book sounded interesting as it likely covers more graphical and interactive QT). I’m going to have to find time to do some sandbox programming with QT sometime soon.
  • Qt5 Python GUI Programming Cookbook ($8.00) Aug
    Combines python and QT as an interesting mix. Python also keeps showing up out there and being able to build QT UI tools with python on the back-end would be quite useful in places.
  • Fixing Bad UX Designs ($6.40) Aug
    I’ve tended to operate mostly in the middle and bottom ends of software stacks in the past. I’m trying to improve my feel for the user experience end of things and the UX books on my list support this effort.
  • Hands-On UX Design for Developers ($6.40) Aug
    A more general purpose UX book.
  • Unity 2018 By Example – Second Edition ($7.20) Aug
    I’d love to find time to build some simple games…particularly if I can combine interesting game play with a networked back end. Unity seems like one of the best ways to make this happen so I’ve been slowly reading enough to find my way around (also one of the reasons for my blender reading).
  • Unity Virtual Reality Projects – Second Edition ($8.00) Aug
    I bought an HTC Vive quite some time ago and while it has been fun to play with (though currently buried in the basement as I sort through years of various items) I really want to do some programming. Unity and VR seem like a good combination…with C# on the back end even better.
  • Hands-On Full Stack Web Development with Angular 6 and Laravel 5 ($10.00) Sept
    I bought this because Angular is high on my ‘web UI’ list and my web hosting here is PHP only. This stuff could give me the ability to break free from WordPress a bit and put some more interesting content on the site.
  • React Cookbook ($7.20) Oct
    React (and Vue a bit) is the other web UI framework that seems to be popular. I’d like to become a bit more familiar with it when time permits.
  • React and React Native – Second Edition ($7.20) Oct
    Another React book with (hopefully) broader coverage and some React Native bits along the way. Web UI for the win?
  • D3.js Quick Start Guide ($2.40) Oct
    Part of a set of four (thus the very low price) covering some odd corners of the web UI universe.
  • Learn WebAssembly ($3.60) Oct
    I have been hearing about webassembly from a few people at work as an interesting technology. Hoping this book gives me some more depth on this front.
  • Learn Three.js – Third Edition ($3.60) Oct
    I’ve been poking at three.js and WebGL for a while now. Another book on the three.js side can’t hurt and for the price I couldn’t turn this down.
  • Mastering SVG ($3.60) Oct
    SVG also keeps coming up at work and while I’m aware of the technology and its basic outlines, I’ve not dug deeper to date. Going to read through this one and see what I can pick up.
  • Kotlin in Action
    Kotlin seems to be an interesting attempt to end-run the conservatism of the Java eco-system. I’ve been looking at it a bit for some time. So far I haven’t found time to do more than some reading though.
  • The Java Module System
    Java 9 feature that I’d like to know a bit more about.
  • The Joy of Kotlin
    More on Kotlin.
  • Web Components in Action
    More web UI advanced topics.
  • Clean C++
    Hoping for some additional insights into modern (2011/2014) C++ usage.

Bouncing around a bit…

…and I don’t think that’s a bad thing.

Still slowly reading VHDL material to get a feel for the language and prepare to work with the Xilinx Spartan-6 board I’ve got at home.

Doing some Win32/C++ investigations to support some file management/archiving tools that have been simmering for a long time but never yet finished. This also drives some C++/CLI work as I’m likely to use C# for the user interface on just about anything I write for home use at the moment.

Continuing Java and Functional programming investigations to support and enhance my current work role and project involvement.

Reading new material (the new Xiling Versal parts look very exciting for a range of applications). Revisiting some older material that I haven’t used all that much (USB hardware level, PCIe hardware level and formal forward error correction).

Kotlin continues to look interesting but not get any time devoted to actually reading the material I have on the language.

A bit of PHP and more WordPress as I look at some networked game stuff and adding another segment to my site here. I’m splitting off the ‘creative’ material from my general purpose blog (which will continue to host my photography for the time being). I’m trying to keep blog entries going as a replacement for my facebook presence. It is a way to keep people I know personally in the loop. This does clutter that area and so I’m setting up pandamallet.com to host more general interest things. So far the new site is skeletal at best, but I’m expecting to keep that moving forward soon.

Still planning to get my RepRap build moved forward. I must find time to read the plans for the Prusa i3 and figure out what my next steps are. I expect that the wooden frame will be right in there and likely followed shortly by buying some filament for plastics and some threaded rod for lead-screws. 

Work continues to eat well over forty hours each week and house projects take a bite as well. I expect this to keep on at least through the early parts of the winter as there’s much on the plate. Hoping to keep the interesting projects in the game a bit along the way.