Andrew Benson

Back in March, 2009 I released some GLSL shaders for Jitter that allow you to perform distortions of an image based on Optical Flow. I sort of came up with it on accident while trying to do something else, and getting distracted during my research. At the time I attached them to a forum thread on Cycling74.com and a lot of people downloaded it, and then they were ported to work in Quartz Composer by Anton Marini, and then people got them to work elsewhere. Since then I have extensively altered the algorithms and tuned the feedback processes for my specific work, but I offer the original patches and shaders here in case it helps anyone else to make something cool. If you do, drop me a line.

Download It!

For the show “Disintegrations” at Johansson Projects (a two-person show with Sebastien Lapointe), I created “Until Every Shape Has Found Its City”, a 3-projector site-specific, interactive video installation based on my “slorp” video-processing software instrument.

While I’m a sort of reluctant Arduino user, I often use them for classes and workshops that I teach because they’re pretty easy to get started with and cheap. Lately, I’ve been a bit disillusioned though with the firmware examples available for using the Arduino with MaxMSP, and wanted to remedy that. I found that most of my students just wanted an easy way to connect sensors and switches to an Arduino and have that show up in Max. While the Firmata firmware is very well designed and offers a lot of features, I have been frustrated too many times by not being able to fix bugs in the code myself. Also, the reliance on the “arduino” abstraction has led to all sorts of problems with students who haven’t learned about file dependencies and search paths in Max.
My solution was to write a very simple and relatively efficient firmware called SensorBox that just gets the input from all the pins on the Arduino and sends it over serial to a Max patch. The communication protocol is hardly innovative, but it should give a good example of ways to efficiently transmit data over serial. The goal was to make everything simple and readable, and to avoid any voodoo, so that others can alter and fix the code as they please.
Also, since many real-world projects require more than 6 analog inputs, I wrote an alternate version of the SensorBox firmware called SensorBoxMux that allows you to configure any analog pin(s) for multiplexed input with an inexpensive 4051 8×1 IC. This offers the potential for up to 96 analog inputs on Arduino!

Download SensorBox firmware and Max 5 patches.

This video was debuted as a looping video installation included in the Uncharted show at Hayes Valley Market, May 2008.

When working on various signal-based sequencing and playback projects, I have found that there are a couple of tasks that required objects that aren’t available in the standard distribution of MSP objects. I have put together a small collection of MaxMSP objects that perform some basic tasks that I find useful for doing signal-driven sample playback of different kinds:

• wait~ – outputs a stored value at the end of a ramp – sort of like a really special sah~. helps to prevent clicking when randomizing or changing parameters of your sampler or granular voice
• shot~ – signal-triggered, uninterruptible ramp output. Length in milliseconds. busy output helps when trying to manage polyphonic setups.
• cshot~ – signal-triggered, uninterruptible sample counter with synchronous ramp output. Length in samples. I’ve always wanted a signal-triggered sample-counter.

Download Mac UB Objects

This small collection is still a work in progress, so any feedback or complaints are welcome.

My Contact Info
email: andrewb [at] cycling74 [dot] com
AIM screen-name: cloudmachine99

Course Outline:::
Jitter Night School 2007
Instructor: Andrew Benson

Course Outline:::

Day 1 :: Intro to Jitter, the Matrix, and Interaction
- Intro to Jitter features
- What is a matrix?
- Using the Jitter documentation (html ref-pages, help files, etc.)
- Jitter data flow
- Matrix as data container
- Audio/Max interaction with Jitter
- Brief intro to Quicktime

Day 2 :: Focus on Video
- Quicktime in Jitter
- Playing QT movies
- Real-time editing in Jitter
- Grabbing input from external video devices
- Recording to disk
- Compression
- Compositing/Blending
- Crossfades
- Keying/Alpha-channels
- jit.op for mathematical blends (additive,subtractive,difference, etc.)
- Filtering, Blurring, and Feedback
- temporal vs. spatial filters
- creating bit maps from images
- convolution
- intro to feedback (named matrices, jit.wake, etc.)

Day 3 :: OpenGL, Slabs, and other advanced topics
- OpenGL lesson
- Deconstructing an OpenGL patch
- the OpenGL matrix format
- applying textures
- blending, layering, and depthbuffering
- Using Slabs for Optimized Video Processing
- What is a slab? What is a shader?
- Optimized transfer of video using UYVY color
- Intro to the provided shaders
- Quick look at a shader program?
- Performance and Live-video concerns
- Using a second display (projector)
- Preloading, RAM vs. Disk-access
- Basic Optimization strategies

Class Patches:::
Day 1 patches
Day 2 patches
Day 3 patches
Jitter Demo Patches

Jitter Recipes:::
Jitter Recipes-Book1
Jitter Recipes-Book2
Jitter Recipes-Book3

Third-Party CV externals:::
Jean-Marc Pelletier’s CV.JIT
David Rokeby’s SoftVNS
Wesley Smith’s Misc. Jitter Externals

Other Awesome Jitter Users:::
Anton Marini – AKA vade.
Joshua Goldberg
Randy Jones
Sue Costabile

Video Downloads:::
Video-Tracking Test Footage
Archive.org- Ephemeral Films,Open-source Video,etc.
VJ Forums Content Finding Forum- Browse the threads here for links to all sorts of short clips in every theme.

—-Not Jitter-specific, but some useful info and inspirations:—-

Traditional Authors:
Giuseppe Arcimboldo: http://en.wikipedia.org/wiki/Giuseppe_Arcimboldo
Giacomo Balla: http://en.wikipedia.org/wiki/Giacomo_Balla
Mikalojus Konstantinas ÄŒiurlionis: http://en.wikipedia.org/wiki/Ciurlionis
Theo van Doesburg: http://en.wikipedia.org/wiki/Theo_van_Doesburg
Marcel Duchamp: http://en.wikipedia.org/wiki/Marcel_Duchamp
Wassily Kandinsky: http://en.wikipedia.org/wiki/Wassily_Kandinsky
František Kupka: http://en.wikipedia.org/wiki/František_Kupka
Paul Klee: http://en.wikipedia.org/wiki/Paul_Klee
Étienne-Jules Marey: http://en.wikipedia.org/wiki/Étienne-Jules_Marey
László Moholy-Nagy: http://en.wikipedia.org/wiki/László_Moholy-Nagy
Piet Mondrian: http://en.wikipedia.org/wiki/Piet_Mondrian
Eadweard Muybridge: http://en.wikipedia.org/wiki/Eadweard_Muybridge
Georgia O’Keeffe: http://www.okeeffemuseum.org/indexflash.php
Alexander Rodchenko: http://en.wikipedia.org/wiki/Alexandr_Rodchenko
Arnold Schönberg: http://www.schoenberg.at/default_e.htm
Alexander Scriabin: http://en.wikipedia.org/wiki/Alexander_Scriabin
Alfred Stieglitz: http://en.wikipedia.org/wiki/Alfred_Stieglitz
Synchromism: http://en.wikipedia.org/wiki/Synchromism

Classic Audiovisual Authors:
Wladimir Baranoff-Rossiné: http://www.baranoffrossine.com/museum/index.cfm?SelectLang=English
Jordan Belson: http://www.centerforvisualmusic.org/Belson
Mary Ellen Bute: http://en.wikipedia.org/wiki/Mary_Ellen_Bute
Stan Brakhage: http://www.iotacenter.org/visualmusic/profiles/brakhagestan
Louis-Betrand Castel: http://www.iotacenter.org/visualmusic/profiles/castellouis
Larry Cuba: http://www.well.com/~cuba/index.html
Tom DeWitt: http://people.wcsu.edu/mccarneyh/fva/D/Tom_Dewitt.html
Charles Dockum: http://www.iotacenter.org/visualmusic/profiles/dockumcharles
Viking Eggeling: http://www.iotacenter.org/visualmusic/profiles/eggelingviking
Oskar Fischinger: http://www.centerforvisualmusic.org/Fischinger
Mary Hallock-Greenewalt: http://en.wikipedia.org/wiki/Mary_Hallock-Greenewalt
Hi Hirsh: http://www.iotacenter.org/visualmusic/profiles/hirshhyfolder
Len Lye: http://en.wikipedia.org/wiki/Len_Lye
Norman McLaren: http://www.nfb.ca/portraits/fiche.php?id=285&v=h&lg=en
Nam June Paik: http://www.paikstudios.com/
Hans Richter: http://www.iotacenter.org/visualmusic/profiles/richterhans
Alexander Wallace Rimington: http://www.iotacenter.org/visualmusic/profiles/rimingtona
Walter Ruttmann: http://www.iotacenter.org/visualmusic/profiles/ruttmannwalter
Daniel Sandin: http://www.audiovisualizers.com/toolshak/vidsynth/sandin/sandin.htm
Harry Smith: http://www.harrysmitharchives.com/
Léopold Survage: http://en.wikipedia.org/wiki/Léopold_Survage
Edgard Varèse: http://en.wikipedia.org/wiki/Edgard_Varèse
Bill Viola: http://www.billviola.com/
James Whitney: http://en.wikipedia.org/wiki/James_Whitney_(filmmaker)
John Whitney Sr.: http://www.siggraph.org/artdesign/profile/whitney/nobio_intro.html
Thomas Wilfred: http://www.lumia-wilfred.org/
Iannis Xenakis: http://en.wikipedia.org/wiki/Iannis_Xenakis

Contemporary Animations:
J. Walt Adamczyk: http://www.johnadamczyk.com/filmvideo.html xvid
Bill Alves: http://www2.hmc.edu/~alves/
Bret Battey: http://www.mti.dmu.ac.uk/~bbattey/
Michael Betancourt: http://www.michaelbetancourt.com/art/telemetry/
Liubo Borissov: http://www.music.columbia.edu/~liubo/bohor05/index2.html
Peter Bosch & Simone Simmons: http://www.fundacion.telefonica.com/at/vida/paginas/v6/eaguas.html
Chris Casady: http://www.naptime.com/flash/artexercises/
Robert Darroll: http://www.iotacenter.org/visualmusic/profiles/darrollrobertfolder/statements
Jeffers Egan: http://www.jeffersegan.com/
Jim Ellis: http://emsh.calarts.edu/~jim/
Robin Fox: http://www.synrecords.com/label.html
Francesco Gaffuri: http://www.francescogaffuri.com/
Andrew Garton: http://www.toysatellite.org/agarton/2004/01/floating_point.html
André Gonçalves: http://www.ctrl.tk/
Robert Hodgin: http://www.flight404.com/blog/
Rumi Humphrey: http://refresh.vu/primera.htm
Samantha Krukowski: http://www.cm.aces.utexas.edu/faculty/skrukowski/
Stefanie Ku: http://www.dioskouroi.org
Anne-Sarah LeMeur: http://aslemeur.free.fr/
José Lopez-Montes: http://www.sibeliusmusic.com/cgi-bin/user_page.pl?url=lopezmontes
Andrew Lyons: http://www.users.bigpond.com/tstex/
Stephanie Maxwell: http://www.rit.edu/~sampph/Works.html
Dennis Miller: http://www.dennismiller.neu.edu/
Bärbel Neubauer: http://www.spiralsmorphs.de/
Scott Nyerges: http://www.nyerges.com/
Richard Reeves: http://www.awn.com/qas/profiles/richard.htm
Joost Rekveld: http://www.lumen.nu/rekveld/wp/index.php
Aaron Ross: http://www.dr-yo.com/
Jøran Rudi: http://www.notam02.no/~joranru/wtca/index.html
Michael Scroggins: http://emsh.calarts.edu/~aka/papers/Absolute_Animation_and_Immersive_VR.html
Robert Seidel: http://www.2minds.de/
Semiconductor: http://www.semiconductorfilms.com/root/soundfilms.htm
John Stehura: http://cyberanimation.tripod.com/index.htm
David Stout: http://nfold.csf.edu/
Rob Switzer: http://www.archive.org/details/Study_One
Ying Tan: http://www.uoregon.edu/~tanying/index.html
Telcosystems: http://www.telcosystems.net/
Shawn Towne: http://web.mac.com/towne/iWeb/shawn/towne.html
Edward Zajec: http://www.dam.org/zajec/

Contemporary Installations/Performances:
Actop: http://www.actop.net/
Richard Bailey: http://www.imagesavant.com/
Benton-C Bainbridge: http://www.benton-c.com/
Steve Beck: http://www.stevebeck.tv/
Coldcut: http://www.coldcut.net/coldcut/
Dienststelle: http://www.dienststelle.de/
D-Fuse: http://www.dfuse.com/
Scott Draves: http://draves.org/
Granular Synthesis: http://www.granularsynthesis.info/
Peter Greenaway: http://petergreenaway.co.uk/
Edwin van der Heide: http://www.evdh.net/
Kurt Hentschlager: http://www.hentschlager.info/
Chika Iijima – Zach Layton: http://www.imagima.com/
Hfr-Lab: http://www.hfr-lab.com/
Speedy J: http://www.speedyj.com/newsite/index.html
Randy Jones: http://2uptech.com/
Skoltz Kolgen: http://www.skoltzkolgen.com/
Ryoichi Kurokawa: http://www.ryoichikurokawa.com/
Ulf Langheinrich: http://www.epidemic.net/geogb/art/gs/ulf/indexgb.html
Golan Levin: http://www.flong.com/
The Light Surgeons: http://www.thelightsurgeons.co.uk/
Jarryd Lowder: http://jarrydlowder.com/
Glenn McKay: http://www.glennmckay.to/
MELA Foundation: http://melafoundation.org/main.htm
Finn & Robin McNicholas: http://ultre.co.uk/
Nokami: http://www.nokami.com/htm_en/01_0.htm
Alva Noto: http://www.alvanoto.com/
otolab: http://www.otolab.net/
Jack Ox: http://www.bway.net/~jackox/
Scott Pagano: http://www.neither-field.com/index.php?page=bio
Power Graphixx: http://www.power-graphixx.com/
Reactable: http://mtg.upf.edu/reactable/
Curtis Roads: http://www.mat.ucsb.edu/~clang/
Ben Sheppee: http://www.eoptica.com/sheppee/
George Stadnik: http://www.photonlightguitars.com/index.html
Jennifer Steinkamp: http://jsteinkamp.com/
UVA: http://www.uva.co.uk/
The Vasulkas Inc.: http://www.vasulka.org/
Leo Villareal: http://www.villareal.net/

Resources:
Center for Visual Music: http://www.centerforvisualmusic.org/
The iotaCenter: http://www.iotacenter.org/

Other Links:
arika: http://www.arika.org.uk/
Ars Electronica: http://www.aec.at/en/index.asp
Center for Brain and Cognition: http://psy.ucsd.edu/chip/CBC2.html
Ernest Chladni: http://en.wikipedia.org/wiki/Ernst_Chladni
Experimental Television Center: http://www.experimentaltvcenter.org/
Fred Collopy: http://rhythmiclight.com/
Color organ: http://en.wikipedia.org/wiki/Color_organ
Richard Cytowic: http://cytowic.net/
Dissonanze: http://www.dissonanze.it/home/home.php
EMAF: http://www.emaf.de/
The Fischinger Archive: http://www.oskarfischinger.org/
International Congress Synaesthesia, Science & art: http://www.sinestesia2007.info/
Lumia manuscripts: http://mssa.library.yale.edu/madid/showthumb.php?id=mss&msrg=1375&msrgext=0&pg=1
Nederlands Instituut voor Mediakunst, Montevideo/Time Based Arts: http://www.montevideo.nl/en/index.html
The New York Digital Salon – Abstract Visual Music: http://www.nydigitalsalon.org/11/index.htm
NOT STILL ART Festival: http://www.improvart.com/nsa/
NOTV: http://www.notv.com/
Optronica: http://www.optronica.org/
Piano Optofonique: http://optophoniqueeng.heatherodonnell.info/
Prometheus Institute: http://prometheus.kai.ru/
Oliver Sacks: http://www.oliversacks.com/
Sonar: http://www.sonar.es/portal/eng/home.cfm
Sons & Lumières: http://www.centrepompidou.fr/Pompidou/Manifs.nsf/AllExpositions/E4B6AD56B6DA4B2FC1256DD600561C77?OpenDocument
Transmediale: http://www.transmediale.de/site/
V2-Institute for the Unstable Media: http://www.v2.nl/
Video Synthesizers: http://www.audiovisualizers.com/toolshak/vsynths.htm
Visual music: http://en.wikipedia.org/wiki/Visual_music
Visual Music: http://www.hirshhorn.si.edu/visualmusic/
Visual Music Marathon: http://www.music.neu.edu/vmm/
What Sound Does a Color Make?: http://www.ici-exhibitions.org/exhibitions/whatsound/what_sound.htm
White Noise: http://www.acmi.net.au/wn_white_noise.jsp

Books:
Expanded Cinema, Youngblood, P. Dutton & Co., 1970, http://www.ubu.com/historical/youngblood/expanded_cinema.pdf
Experimental Animation, Russett – Starr, Da Capo Press, 1976, ISBN 0-306-80314-2
The Sound of Painting, von Maur, Prestel, 1999, ISBN 3-7913-2082-3

Sons & Lumières, von Maur – Rousseau – Cooper – Levin – Lista – Kahn – Duplaix – Kihm – Szendy, Éditions du Centre Pompidou, 2004, ISBN 2-84426-244-9
Visual Music: Synaesthesia in Art and Music Since 1900, Brougher – Strick – Wiseman – Zilczer, Thames and Hudson, 2005, ISBN 0-500-51217-5
audio-visual art + vj culture, Faulkner/D-FUSE, Laurence King Publishing Ltd., 2006, ISBN 1-85669-490-9

I will be in a show called “Excavations” at Johansson Projects Gallery in Oakland,CA. The show opens May 12th, with a reception that evening and also another one on June 1st during the Oakland Art Murmur, 6-9pm. The gallery will also be open Thurs.-Saturday, noon-6pm, during the rest of the show. This is the first show at the new gallery, and will include work by eight other artists in various media. My contribution to the show is 16 paintings, 9 video pieces, and one felt and thread landscape installation. For more info, check out the gallery website.

Today I posted an article over at Cycling74.com about using joysticks with MaxMSP. Toward the end of the article, I discuss ways to alter the electronics of a USB joystick for custom controllers.
I had been meaning to write this piece for a long time, and I was assigned the task of writing an article for the website, so I figured this would be a good solution.

Since I’ve been working for Cycling ‘74, I’ve taken a bit of time out of my busy Support/Testing day to put together these “recipes” based on whatever interesting video trick I happened to be working on that week. Many of these are proof-of-concept types of things, demonstrating that certain things can be done, and one possible approach to a problem. More often than not, the concepts are drawn directly from a project I’m working on, and comprise a collection of my secret processes for doing video and audio work.

Link

The spazkron control is a gestural, wearable performance instrument built around a hacked USB game controller and 2 piezo contact microphones, as well as a custom sound synthesis patch built in MaxMSP.

The most conspicuous aspect of the device is the pair of long wooden dowels that extend from the abdomen of the performer. These are attached to patches that contain the analog joystick elements from the game controller. The sticks function as long joysticks that protrude from the body. The 4 axes provide analog values that are mapped to various functions in the audio synthesis software.

The contact mics are embedded into felt wristbands that are worn by the performer. Sound is triggered by striking or scraping these contact mics against the sticks. The Max patch performs envelope following on the incoming signals. The resulting envelopes are used as parameters for various sound generation functions, in addition to volume control. This offers the performer a great deal of expressivity and subtlety.

In building this controller, I was interested in creating something that required a certain amount of physical engagement to make it function. I am primarily interested in performing complex, spazzy noise stuff, so I needed a controller that expressed that sort of physical movement. Because of the nature of the device and its relationship to the body, the performer must twitch, contort, and flail in order to really engage the full complexity of the system. I was specifically not interested in a controller that was polite, reserved, and cool. I wanted my device to force clumsiness and absurdity upon the situation, which is something that is often missing from experimental electronic music.

Jeff Kaiser has a brief blurb about a recent performance here