bzztbomb

Doctrine of Error

These images were generated with a pretty simple program. It takes an input image and compares it against the current painted image. The program generates a new brush stroke and the new error is calculated between the input image and the painted image, if the stroke reduces the error it is kept, otherwise it is rejected. Final image here

Tank Man – Painted with the Doctrine of Error

I like how this one turned out, the source image of Tank Man was a bit noisy and I think it interacted with the algorithm in a cool way. Final image here.

I wrote this program in Racket. I’ve been meaning to learn a LISP or Scheme based language for a long time. I thought this program would be a fun way to start. The brush strokes use a technique called Globs which basically connect to circles together with splines in a way that it has first-derivative continuity with the circles. It’s a nice way to generate brush strokes. Final image here.

It was inspired by the Create Applications of Deep Learning class and Ben Garney’s blog posts about writing a proof of concept video chat system. In both cases, the technique of defining, measuring, and reducing error is the core concept that makes it all work. Final brain here.

In the future, I’m planning to apply this to video. It’ll be straightforward to take the previous frame of a video and use it as the starting point for the next frame. I think the effect could be cool. I might also port a simple version of this to a more popular language and give a DorkbotPDX workshop on it. Final clocks here.

iOSBNIZ

iOSBNIZ is a port of IBNIZ by viznut to iPhone and iPad. This allows me to program little audiovisual hacks on my iPhone with minimal typing. IBNIZ features a VM with opcodes that are one character long, ideal for not hitting too many buttons. It also has a FORTH like stack which is a fun puzzle to play with. The VM calls the program for each pixel and pushes time, x, and y onto the stack. Your program uses these values to generate the visuals (and audio). You can also write a full program and ignore this loop as well.

iOSBNIZ is available on the Apple App Store. Download and have fun!

Source code is avaiable on Github

iOSBNIZ

IOSBNIZ Features

This Is How We Disappear

I’ve been meaning to document my work on this project for about year now. TBA 2014 reminded me it is time to actually do it! So here it is:

For the past year and a half I’ve been working with a dance company called BOBBEVY. I’ve been creating graphics that go along with the dance performance called “This is how we disappear”. Here’s a review at Portland Monthly.

Jesse Meija was doing music and got me involved with this project, I’m very grateful!

Version 1

Effects for the first set of performances:

In order to accomplish this, I wrote a piece of software that would do the animation and handle tracking the dancers. There were two versions, the first version was used to perform a few times. Notably Dance+ in 2012 and as part of Experimental Half Hour XXXVII. It consisted of the following pieces:

This version worked ok. Before I started using Control, I had been triggering all of the sequences with keyboard commands. It worked fine, but I had to have a cheatsheet that told me what keys did what. Also, each command just mutated the state of the program, so if you triggered things in a different order you’d end up in different states. This made some rehersals hard, because it was difficult to return the graphics to a previous state. However, with Control, it became easier to use the software. BOBBEVY performed in Milwaukee without me and was able to use the software just fine! For Dance+ the Kinect refused to work in the studio, I think because the temperature in the room was so high. So I ended up “drawing” the dancers with a multi-touch interface in Control.

For the particle effects that followed the dancers, I ended up using blob tracking and distingugishing blobs based on distance away from the Kinect. I liked the stateless design because the dancers would move in and out of view of the Kinect and I feel that keeping track of them properly would have been a nightmare. This created some surprising benefits though. The swarms move between the dancer when their relationship to the Kinect changes and it created some really nice animations. Also, this piece has a lot of tension between the dancers and the particles ended up expressing some tension when the dancers were about the same distance away from the Kinect.

Version 2

Additional effects for the second set of performances:

This second version of the software was used to perform at NW New Works Festival 2013 and at TBA 2013.

For the second version of the software, I used the following new pieces:

The timeline solved many problems for me. It took what I used to have hard code in the application (fade times, animation speed, etc) and moved it to a data format. The editing GUI was nice to have as well. A new version of Cinder that made using multiple displays easier to use was really nice to have as well. I didn’t need to mirror my desktop screen anymore, which meant I could display debug and other helpful info on my screen. The QuNeo also allowed me to directly control ramp parameters which meant I didn’t need to rely on predetermined fades as much. This also allowed me to be more engaged with the visuals which was really fun. I think the trick to this will be finding the right balance between direct control and triggers to presets. It is probably the same balance electronic musicians search for.

The newest effect for the second run was the projection mapped dancers. In order to accomplish this, I was going to have to find the dancers with the Kinect and then project onto them as close as possible. I used the vvvv patch from here as a starting point to learn how to calibrate my projector with the Kinect. In the end, I wrote my own calibration code because it fit the setup workflow a bit better.

The projection mapped dancers worked pretty well. I was really excited to see them turn into just an indistigushable mass at moments and then turn back into dancers the next. I think this is what projection mapping should do: transform objects and confuse you, then bring you back to reality. I hope to do more of this in the future!

Church of Robotron Sermon

On March 25th, 2013 the Church of Robotron gave a sermon at DorkbotPDX 0x0A. It focused on the mobile Church of Robotron installation we did at Toorcamp 2012. James gave live sermons interspersed with presentation snippets about the installation. We projected our faces onto old TV’s, an homage to Dr. O’Blivion and the Wizard of Oz. I mutated the Party House project to support webcams and applying shaders to the input stream. I was able to control the video source and shaders at realtime during the sermon. We also had slides. I think it was a nice mix of a live human being, psuedo live humans being projected, and standard slides. I hope to do more sermons in the future!

NW New Works Festival 2013

Doing live visuals with BOBBEVY this weekend in Seattle, WA as part of the NW New Works Festival 2013. Check it out!

Use MAME's debugger to reverse engineer and extend old games

Use MAME’s debugger to reverse engineer and extend old games

For the Church of Robotron’s installation at Toorcamp 2012, we needed to be able to trigger physical events when game events happened in Robotron 2084. A quick summary for context:

We choose to use MAME’s debugger to detect game events and notify other pieces of software when they happened. This is a quick tutorial for others (and a reminder to ourselves) if you’re interested in doing similar things. We’re going to find out how to detect player death!

Prereqs:

Here we go!

Start mame in windowed mode with the debugger active:

mame -window -debug robotron

You’ll see two windows appear, one is the emulation, the other is the debugger window. You can type “help” at the console and get good documentation on the debugger capabilities. You can also click on the down arrow to access a menu that allows you to run the emulation and later pause it at different points.

debugger

Memory dump

The first thing to do to find the memory location of the number of lives the player currently has. One way to do this is to take advantage of other peoples work. ;) Sean Riddle has a great site with a lot of this information already available! The other way is the manual way. Let’s do that!

Now we have dumps of the game with 5 lives and 3 lives as the default. Let’s use a diff tool to see what changed in memory! On OSX, you can use Filemerge or standard old “diff”. Scanning through the differences, we don’t see anything that correlates to 5 lives and 3 lives. But we do see two spots that seem to correlate to 4 and 2 lives:

2722,2727c2722,2727
< AA60:  18 07 FF 06 5F 04 1D AA 58 39 CD 02 00 00 00 00  ...._...X9......
---
> AA60:  18 07 FF 06 43 03 E9 AA 58 39 CD 04 00 00 00 00  ....C...X9......

This address is AA6B.

3039c3039
< BDE0:  00 00 00 00 00 00 01 00 00 02 50 00 02 01 14 0A  ..........P.....
---
> BDE0:  00 00 00 00 00 00 01 00 00 02 50 00 04 01 14 0A  ..........P.....

This address is BDEC.

Setting watchpoints

Let’s see if we can test our guesses!

Go back to the debugger:

We’ve found the memory location for player lives! We can also take note of the instruction pointer address. It’s at 27AC, let’s set a breakpoint here.

Setting breakpoints

Continue playing. You’ll notice the breakpoint is hit for every player death except the last one. That’s a bummer! Let’s walk up the stack and figure out a better spot. On some CPU types, you can do just that, but with the 6809 that’s powering Robotron you can’t. So we’ll turn the trace file, run the game until we hit our breakpoint, and then close everything. We can step back up from the bottom of the file and see if there are interesting addresses to set breakpoints on.

Trace file

Open the trace.txt file in an editor. Go all the way to the bottom. You’ll see a nice trace with repeated blocks called out. I use these blocks as starting points for new breakpoints. If you scan up the file, you’ll see repeated addresses of DC56. Skip above those, they’re not interesting. We want to find the first branch of code that doesn’t repeat. Just scanning here are the addresses that seem interesting to me:

0xD676, 0xD8BC, 0xD1ED, 0x30FE

Let’s set some breakpoints:

Start running!

You’ll notice D1ED gets triggered alot, so disable that one with the “bpclear (breakpoint number)” The same thing happens to D8BC and D676. But, 0x30FE seems to work!

Summary

I hope this helps people get used to the MAME debugger. After you have this information, you may want to do something with it. We choose to broadcast game events over UDP. You can see how by looking at our Github repo here.

PRAISE THE MUTANT SAVIOR!

Party House

Party House is a projection mapped dollhouse that was quickly thrown together for the DorkbotPDX Open Mic Surgery event. It was written in a few days using the Cinder library. The source code is available here: PARTY HOUSE REPO.

It was super fun to perform. It wasn’t a complex or hard thing to do, but sitting up on stage pumping my fist and hitting space bar over and over again to the beat was fun. I’d definitely like to do more of this kind of “work”.

I had originally followed this Madmapper tutorial on mapping a building with madmapper and creating a video to go along with it in After Effects. It was a great introduction. I ended up running into problems in my case because the doll house had lots of overhangs and different planes to work with. Maybe if you’re projecting on a huge building, the building dwarfs the projector by enough that you can treat it as one plane. But in this case, I had to create a bunch of planes to make everything line up nicely.

This is also the first time I’ve used After Effects at all. It was nice to just make some videos and them layer them on the fly during my set. I definitely want to spend more time learning After Effects for future projects.

I ended up writing my own software to do the mapping because I’m a hacker and just wanted to. The code isn’t that great, but I do think the EditorViewport and ControlPoint classes are nice enough to re-use in future projects. I also updated the code to use the newest branch of Cinder, so it now supports multiple windows. It was also fun to start using C++11 features.

Church of Robotron

The Church of Robotron is a future looking group that is attempting to save the last human family. We did a large pop-up church installation at Toorcamp 2012. It had the following features:

I hacked MAME to expose debugging events (breakpoints, memory watchpoints) to clients via sockets. I also did some reverse engineering of the ROM to find out game events (deaths, high score, etc). To do this, I built upon the work of the great Sean Riddle. I also used OpenCV to capture the players faces when they died which were posted to the high scores page. We had about 15 people working on various pieces of the project. It was great to see it all come together and it was great to see people’s reaction to it!

Source code is available here. More tech details will be written at some point. I’m planning to write a quick post about mame hacking at some point soon.

BOBBEVY, This is how we Disappear

(Seek 19 minutes in)

“This is how we Disappear” is a 25 minute dance performance created by the BOBBEVY dance company. I used a lot of random technologies: Cinder, Microsoft Kinect, libfreenect, OSC, and Control to create realtime dancer reactive visuals to go along with the performance. We performed it quite a few times and my favorite time was at Hollywood Theatre as part of Experimental Half Hour. It was great to see the graphics projected so huge!

I’ve learned quite a bit about what’s required to do visuals for a performance from this project. It’s definitely influenced everything I’ve done afterwards. I’m hoping to do more of this type of work in the future.

Some post-mortem notes:

Using the Kinect is a performance space is a pretty harrowing experience. If there’s a lot of reflection or IR noise in the room, the Kinect can just die. I think there are some tweaks you can do in libfreenect to allow one to continue to get data, but it’s likely to be too noisy to work. I ended up creating a fallback that allowed me to “draw” the dancers with my figures with a Control sketch. But that method isn’t ideal.

I initially triggered everything with keyboard keys, but it became hard to remember what did what after a while (even for a powah user like myself). Have an iPad user interface was much nicer and allowed the BOBBEVY crew to run the show themselves when they performed in Milwaukee. In theory, this would also let me trigger the cues far from the computer which would give more options for where the computer and Kinect go. In practice, the UDP packet loss when doing Adhoc networking seemed to high to rely on this. It’d be nice to have a TCP transport as an option in the future.

I also added Bonjour discovery to everything, this made it much much easier to get everything configured before a show. I recommend this to people doing multi-machine work.

DorkbotPDX Exquisite Corpse

dorkbotpdx

I contributed to this project. I can’t reveal details until we reveal the whole project!

(PS: We did complete this project)