Category: Computer Related

Posted on

Onwards to Linux C++ OpenCV and Capturing Some Frames

I proven that the cameras can run on Ubuntu and the RPi. I found a page with classic Unix/Linux style install instructions. I’ll be working on getting this set up on my biggest RPi machine and take a look at building code to red from multiple cameras and stream the data to a host. If I can run two cameras on a single RPi then I should be set.

I’ll probably also look at doing something similar for the RPi cameras on the RPi-2 machines. That might add a couple of additional cameras to my set.

I’ll then move on to building a simple LED beacon and look at some simple camera calibration code on an appropriate host.

Trying to build the OpenCV package on one of my RPi 3 machines. I think I’m running into heat issues. I’ve switched to a board that I can keep open and has a heat sink on it. Hoping that may be enough. I’ve also dropped the build scripts onto github to make them broadly available.

Pulling packages on this machine now.

Now I’ve got a fan blowing. I see from some web pages that the RPi is supposed to warn and throttle when the temperature spikes…didn’t see that with my black and silver RPi 3…it seemed to halt completely after a short span. Keeping this one cool up front and we’ll see how this goes.

Interesting…it looks as if the scipy build is eating all available physical memory on the RPi board (882 MB of 923 total). Nothing moving on the machine…not even the mouse cursor.

Ah, reference here to bumping the swap file for the RPi.

Monday morning dawns and it appears that I have a raspberry pi that is loaded up with an installed build of OpenCV. No time this morning to test this but tonight I’ll run through a few simple tests and then probably run the same process on one of my Intel NUC machines (should go faster and easier) to get a decently powerful system up and working with the same version.

Posted on

PS3 Eye Cameras Came in and RPi Machines are Going

Yesterday my five additional $6.00 PS3Eye 60 FPS cameras arrived. I’ve pulled my RPi machines and my two Ubuntu based NUC machines out and started checking out the cameras and making sure the systems are up-to-date.

RPi 3 machines and various supporting parts.

The NUCs generally get more use and thus needed less updating. I found that using cheese I was Immediately able to run the cameras on both the Ubuntu NUC machines and the RPi boxes. Even the older RPi-2 machines ran the cameras…I’m not sure that the performance on those will be sufficient to make them useful though. They do both have RPi internal cameras connected though so that may be of interest. I’ve got to look at how to access those cameras from C++ code and see what sort of performance they have (and test them out to see what they can do).

The RPi-2 machines with attached cameras…

Here are the cameras (the one sample I bought for testing and the five others that just came in). I may pick up a couple of spares to make sure that I have what I need in the longer run…at $6.00 each that isn’t a big deal). The beer pong balls look like the perfect size for a light diffuser on an LED…I’ll have to rig up a few LED/resistor/battery sets to try that out sometime soon.

Cameras and diffusers waiting to be unpacked.

On Sunday I’ll look at getting programmatic control of these cameras. If I can acquire images from two cameras at the same time and stream the data out over a socket, I’ll be ready to rock with these things.

Still to be managed is getting the cameras rigged to mount on light stands and getting three more light stands with ball heads to set them on. If the RPi-2 boards look interesting with their integral cameras then I may need some additional mounting options, but initially three to six cameras should be a good start.

Posted on

Got the *.fbx SDK installed and my RPi machines out and booted up…

I’ve installed the filmbox SDK on my home machines so that I am in a position to play with programmatic manipulation of *.fbx files. This feeds into the motion capture experimentation I’m doing and should be generally useful at times.

I also dug out my three RPi 3+ machines last night and got one of them booted up and connected to a PS3Eye camera…without actually taking images…just observing the power light go on and no other negative effects. I’ll likely move further over the weekend to try getting actual image capture working there.

I did find a github project that claims to be able to run one of these 60 FPS cameras from an RPi. This project appears to be motion sensing related, but I expect it can act as a source of sample code for my longer term purposes…

Posted on

Sample Cameras for MoCap Experiments Came In

My sample cameras came in. I ordered two cheap webcams, a Sony PlayStation 3 Eye Camera and a Fosmon USB 6 LED 1.2 Megapixel USB PC Webcam as these were both under ten dollars and looked potentially interesting.

The Fosmon camera looks too slow and low on image quality to be useful for my purposes here so I’m shelving that one. The PS3 eye camera is challenging to interface to a PC, but at $6.00 per camera, I’m inclined to spend some effort here. There do appear to be Linux drivers out there and I’m going to look at a RPi mediated option with these (I’m willing to invest $30.00 for five more cameras to give this a try).

The other option that was presented as viable are Logitech C922x Pro Stream Webcams. I have two Logitech C930e cameras currently…these won’t run 60 FPS but have comparable image quality and a wider field of view. At $70.00 each, I won’t be picking up six c922x cameras any time soon, but I may purchase on or two if all goes well to see what they are capable of. They’re almost certainly better cameras than the PS3 Eye devices, but rather pricey.

I’m also picking up some CR2023 batteries and battery holders along with a box of LEDs and some 19mm ‘ping pong balls’ to try as diffusers.

This is all an experiment, so it may come to nothing more than an interesting diversion. The commercial motion capture options are far too expensive to be viable as a hobby thing and I’m expecting to have some fun putting this together and seeing where it can go.

I would be particularly happy to see an RPi solution work out as having one pi board for each pair of cameras (or even for each camera at a stretch) with an Ethernet back-haul to a full fledged PC for processing would be a flexible and relatively affordable solution with pretty good scalability to more cameras as well.

Posted on

3D Asset Export and Sharing

There have been concerns expressed about getting from a point cloud to imported animation data tied to a skeletal model.

I accept that this may not be trivial…particularly tying a set of points in a moving cloud to control points for bones in a skeletal animation.

Collada

It does appear that there are defined interchange formats (I’ll have to look and see if these are supported bu Unity and Blender) that may make this more straightforward. Being able to export the skeleton for an animation and then inhale it into a processing application seems helpful. Being able to tag points in a cloud for consumption by Unity seems even more helpful.

I looked here and got a pointer to collada which appears to be getting developed by Khronos at this point. This appears to be a general purpose, open, XML based standard for interchange of 3D information. There is also an OpenCollada code archive out there…that link appears to be dead…but it is on GitHub and a project home page.

Wikipedia article here.

FBX

Filmbox is a more proprietary option with information at Wikipedia, Blender, More Blender, and AutoDesk (who created it and still owns the format). The AutoDesk link connects to SDK downloads that support C++ and Python manipulation of these files.

Posted on

Unity Stars and Textures

This weekend, Malcolm and Sam came over for a day to move our various Unity investigations forward.

Malcolm made some very nice pieces of furniture and architectural bits. I think the end result has much more feeling of place and I’m moving forward from there.

The Cluster Control Room

Sam was working on his own project and looking at getting his motion working properly on Daydream with the Daydream controller.

I added some interactivity to my prototype. Attaching a ‘stick’ to the controller and hanging a small sphere with a collider attached makes it straightforward to select objects by touching them with the stick. It took a surprisingly long time to find the appropriate code to implement haptic feedback when a star is touched. I’m still looking at adding a short positional sound queue on touch as well.

The code lives on my github account here. This supports Vive only (I think) at the moment. It certainly supports Vive best. At some point I’ll likely look at making this work with DayDream and flat screens.

Some Sample Textured Spheres…

I grabbed some textures from around the house and yard and slapped them on some random spheres and capsules. Those are here and there are separate branches for DayDream, Vive and flat screen (master). The Vive version looks much better than the DayDream version and I expect that is to be expected.

I spent some more time chopping up texture tiles. Not a lot of time to clean them up though. I really need to better understand the impacts of the various extra texture functions that are available. I think there are some options missing from the default material such as displacement maps that can genuinely move vertices. Lots to learn along the way and getting there with the Vive is pretty cool.

Posted on

Watching more Unity Video Sessions

I’m looking for more best development practices ideas, future direction information and some bits on existing subsystems so I expect to spend a chunk of today watching sessions from 2018 ‘Unite’ conferences and related video segments. I’ll likely add links and comments here as I run through them…

Yesterday

Yesterday I watched a bunch of videos roaming around the unity event system (a bit disappointing), possible enhanced event systems (pretty interesting), futures (ECS and the job system) and some singleton/scriptable object bits. I ended with a unite session that also touched on some best practices areas where I want to see more.

Unite Austin 2017 – Game Architecture with Scriptable Objects – A talk by game developer Ryan Hipple from Schell Games. It covered quite a bit of best practices material with extensive suggestions for ways to leverage tools in Unity to provide developer accessible dependency injection in game internals. This talk got me started thinking that there might be a decent amount of best practices information available out there.

Unity3D Mistakes I wish I’d known to avoid – This talk covers a number of built in facilities that can provide capabilites without scripting involved that the presenter has seen many people implement in script. Understanding the tools that are already available in Unity as it currently exists is clearly worthwhile and likely a bit challenging as this is also clearly a moving target.

Unity ECS Basics – Getting Started – With 100,000 Tacos – A quick demonstration of the entity component system. This does seem like a nice new facility to accelerate things in Unity games. At this point (not sure about in the 2019 beta) it seems to lack physics and some other functionality that are presumably in the works. This and the jobs system appear to be the next frontier for unity game development.

Today

I expect to poke into scriptable objects, perhaps a little ECS and Job stuff, as much best practices advice as I can find (likely a bunch of unify sessions) and some serialization (particularly saved game issues where the saved game can be sent to a web server) and web remoting if I get to that.

Introduction to Scriptable Objects – Looking at the Unity provided introduction to scriptable objects. These seem to be largely game objects without transforms essentially. Data wrappers that can hold game related information without pulling in the rest of the overhead of a full game object.

Interesting serialization blogs here and here and here .

Serialization in-depth with Tim Cooper: a Unite Nordic 2013 presentation -Hopped over here from the above presentation…older but still interesting. Shows some of the attribute controls that make finer grained serialization possible.

I suspect that much of this material has evolved since 2013 but I’m looking at this as a baseline and some visibility into the back-ends of things. This talk provides lots of detail about the advantages of using scriptable objects for stored data in a unity project. They are far more reliably serializable than game objects and editor to game transitions won’t properly save and restore game objects in many cases.

The presenter is also discussing custom editor UI implementation. Look at CustomPropertyDrawer. Need to look into assets and SubAssets, [serializable] and asset trees. AddObjectToAsset(…). Need to remember HideFlags. Interesting items: HideAndDontSave. Selection.ActiveGameObject().

Best Practices for fast game design in Unity – Unite LA – More best practices. This one is oriented towards fast prototypes of mobile games so I’m suspecting that it won’t apply well to my personal use-cases.

Mentioned AssetForge as a way to build models. Potentially interesting. Houdini. mentioned. Jenkins builds for Unity games also mentioned. Cinemachine Odin Amplify Shader Effectcore.

Unite 2016 – Overthrowing the MonoBehaviour Tyranny in a Glorious Scriptable Object Revolution – Talk by the head of sustaining engineering at Unity. A detailed look at ScriptableObject. I feel as if I’m definitely getting closer to understanding how things hang together…not there yet, but closer.

Scriptable object don’t get most callbacks. Same type in the core system but cannot have gameobject or transform attached. Scriptable objects don’t have to get reset during play mode transitions. OnEnable, OnDisable and OnDestroy only. ScriptableObject.CreateInstance<>(). AssetDatabase.CreateAsset() and AssetDatabase.AddObjectToAsset(). [CreateAssetMenu]

Empty scriptable objects as extensible enums. Derivation from a common base? JsonSerializable by JsonUtility and as built-ins. Look at LoadAsset<>(“”); FronJsonOverwrite(json, item); Can patch part of an object with this.

Example of building a robust singleton ScriptableObject that recovers properly on reload. The demo project here looks kind of interesting…on bitbucket.

StartCoroutine() looks interesting.

Interesting session. Getting enough food for thought to make it worth mucking about with some code soon.

Unite ’17 Seoul – ScriptableObjects What they are and why to use them – The next year’s talk on scriptable objects…

AddComponent() for monobehavior. Monobehavior is always attached to a gameobject and transform along with any other components attached. [Range(min, max)] as ranged variable. Look at CreateAssetMenu attribute. Custom property drawer.

ScriptableObject.CreateInstance()

AssetDatabase.CreateAsset() or AddObjectToFile()

Diablo: A Classic Game Postmortem – A nice break hearing the lead developer of Diablo talk about the development of the game…

I also watched a session on game pitches that I never expect to have a use for. Playing with games in my spare time appeals, but I can’t currently see any chance that I’d want to try working in the industry.

Interesting blog post What is the purpose of ScriptableObject versus normal class? Talks a bit more about scriptable objects.

Fast worldbuilding with Unity’s updated Terrain System & ProBuilder – Unite LA – Interesting items here as terrain and such makes for interesting tactical and strategic games. Assets, tiles and patches oh my!

A rather cool set of tools I had not encountered before. Looks like an amazingly quick way to create credible terrain. Got to look at some of the relevant demos as well.

Probuilder looks like an interesting tool for creating geometry and painting geometry. All free asset store items. Probuilder, Progrids, Polybrush. Currently available in the package manager except for the brush. Looks as if this provides a good bit of capability to create and modify geometry in the smaller scale than the terrain tool. ProBuilder-API-Examples looks like the next step to allow this stuff to happen at runtime.

Introducing the new Prefab Workflow – Unite LA – The first in a set of ‘Editor Sessions’ grouped on youtube. I don’t expect to have time tonight to watch all eight of these, but hopefully I’ll get to them in the near future.

These are the workflows that exist in 2018.3 that we’re currently working with so this is likely to be very relevant.

First item is true prefab nesting support. Variants were mentioned, but I think they’ve been around longer than the nesting support. Prior versions appear to have exploded prefabs within prefabs when apply was clicked. Variants as derived prefabs essentially…the overidden parameters stay overidden but changes to the base propagate.

What are prefab models? Blue cube with a page symbol. Looks like they’re an early import thing for models coming in from outside. Ah…looks as if variants allow external meshes to be imported in a way that they can still be replaced after the prefab is rigged.

Improved Prefab Workflows: the new way to work with Prefabs – Unite LA –

Material I Don’t Expect to get to Tonight…

Posted on

Unity Cinematic-ish Tools

I was bumping around youtube and the Unity site and ran across references to the Timeline feature and the free Cinemachine plugin.

Timeline

Timeline has an intro video here and allows recording of scripted motion with triggered playback. It seems to have originated as a way to play out cut-scenes, but looks like it would also be useful to provide background activity in a scene with objects and ‘extras’ moving about to provide additional color.

I probably won’t play with this too much this weekend as I’ve got some work items to get a few hours on and I want to look deeper into the areas that my VR board-game needs.

Cinemachine

Cinemachine is a free camera management plugin. This looks like a wonderful way to manage a dynamic camera following a moving object. You lock the camera to a transform and set its parameters to keep the following process fluid. The introductory video is here. This one is also likely a ‘later’ item as I’m not currently planning much in the way of free motion or background animations in the cluster project.

Transitions with Animation

Interesting video showing how to use animations to create no-code transitions.

Scriptable Objects and Game Architecture

Watching this video on scriptable objects and game architecture. It seems as if I could stand to know more about these and their place in unity game architecture. I’m actually interested in non-unity assemblies as bolt-ons for a unity game. These would allow ‘business logic’ to be provided externally and tested using nUnit or something similar while providing services to the game as a black-box-like component. As they’re C# they should be readily debuggable along with the built in C# code in game objects and such.

This is one of the big steps to take…I would like a better feel for how the unity system works best. Building flexible, scalable things is always more fun…

Event System Ideas

Here is a talk that details an event system where the event grouping is handled by the type of the event object. The listener subscribes to notification based on the type thrown and the core event system throws based on the type passed to the event firing method. Keeps things clean and moves type mismatch errors cleanly into compile time.

There may be downsides (and I need to look more closely at the current unity event system as this may be similar to what we have in the 2018.3.6f1 version…

I think I’m going to wind up watching many more of these sessions to get a better feel for what works well in unity-land at the architectural level and from a finer grained point of view.

Posted on

Working Through the Technical Side of Unity

Sam, Malcolm and I have been diving into the more technical side of Unity development for the last couple of weeks. I expect this will go on for a while to come as well. There will probably be items on this side now and again as the Cluster game hits plateaus along the way and there are graphical bits and game play decisions to talk about.

Currently the technical content is being written up on my career blog. Perhaps a less than ideal naming choice as it gets most of the hard technical content yet I’m never likely to take a job in the games industry.

I’m currently working out a new scheme for partitioning up the pieces of the game and making the cross platform support (Daydream VR, Vive VR and flat screen PC) workable. I’m also likely to rebuild some of my spaceship assets as run-time created prefabs rather than manual prefabs.

Proudly powered by WordPress