The Wekinator is free software that runs on Windows, OS X, and Linux. Download the version for your operating system:

    For Windows or Linux:

        Download here

        Try doing a walkthrough to ensure everything is set up properly.

    For OS X (including Lion and earlier):

        Download here

        Try doing a walkthrough to ensure everything is set up properly.

Download the example code bundle

It’s here: www.cs.princeton.edu/~fiebrink/NIME2012Workshop/examples.zip.

Feature Extractor Examples & Instructions

The gestural, audio, or control inputs generated by the user and sent to Wekinator are called “features.” Wekinator has built-in support for the following inputs:

1. •Trackpad/mouse (OS X only, pre-Lion),
   

2. •Laptop built-in motion sensor (OS X only, not available on Macbook Air)
   

3. •HID devices such as joysticks and other standard USB game controllers
   

4. •Webcam: basic edge detection & color tracking (via Processing, requires webcam
   

5. •Basic audio features (RMS, spectral centroid, spectral flux, spectral rolloff, FFT bins)
   

Additionally, you can send features to Wekinator using OpenSoundControl. Wekinator expects a stream of OSC messages (at any rate) to arrive at port 6448, using the message string “/oscCustomFeatures”, where every message contains n floats. The order, number, and meaning of all floats must be the same for every message. For example, to use two Max/MSP sliders as input features, Max would send OSC messages containing two floats: the first for x and the second for y. Then, to tell Wekinator to receive the OSC features, once you’re on the “Features Setup” tab, you’ll check the box for “OSC Custom Feature Extractor” and enter the number of features, n, in the corresponding box.

Feel free to start with example feature extraction code for the following inputs/environments, which are enclosed in the example code bundle linked above:

1. •Max/MSP: Simple 3-slider example in SimpleMaxFeatureExtractor.maxpat (3 features; requires the OpenSoundControl Max object from here)
   

2. •Max/MSP: Bark coefficient audio analyzer in BarkExtractor.maxpat (3 features; requires OpenSoundControl and analyzer~ objects)
   

3. •Processing: ProcessingSimpleColor and ProcessingSimpleColorLion extract average webcam input color (3 features; can edit code to extract color from grid cells across webcam input). (Requires oscp5 library; Lion version also requires codeanticode.gsvideo library.)
   

4. •(OS X Lion doesn’t play well with Processing Quicktime, hence the need for a Lion-specific version)
   

5. •Kinect on OS X, using OpenKinect/OpenNI + Processing: See instructions here.
   

6. •Kinect using Synapse: Alternatively, you can download Synapse then use another program, e.g. as the SynapseMaxRouter.maxpat, to route features to Wekinator.
   

7. •Wiimote: Osculator (free trial version available here) works great for sending Wiimote data on OS X. The example bundle contains osculatorWiimotesettings.oscd, an Osculator settings file that sends raw x, y, and z accelerometer values to Wekinator (3 features).
   

8. •Osculator (download above) also works great for reformatting arbitrary MIDI or OSC messages to Wekinator’s expected format, and passing them to Wekinator. Basically, for all inputs but one, save the input as a “variable” by setting event type to “Variable” and value to a unique index from 0 to n (if you have n+2 total inputs). For the other input, set the event type to “OSC Routing” and put arg[0] and all variables (i.e., var[i] for all i from 0 to n) in the message in whatever order you want. Ensure each of these is sent as a float (will show up with the f: prefix in the Arguments list). When this last input updates its value, it will trigger the sending of the OSC feature vector to wekinator.  (Just look at the example settings file osculatorOSCemoteTouchpad and it will make sense, really.)
   

9. •touchOSC, OSCemote, and similar OSC apps that run on a smartphone or iPad can similarly be routed through Osculator. See osculatorOSCemoteTouchpad.oscd for an example Osculator settings file for the OSCemote 2D touchpad (sends 2 features: x and y position of 1 finger).
   

10. •Wacom tablet: Nicolas d'Alessandro wrote a nice patch that transforms Wacom into OSC messages, included in the bundle in wacomToOsc/wOsc. Run this with wacomToOsc/osculatorWacomSettings.oscd (7 features). Alternatively, if you have a Wacom object in Max/MSP, you can adapt one of the Max examples to send your Wacom values to Wekinator.
    

11. •If you’re on Windows and want to use MIDI or some other OSC device as an input, your best bet is to make a Max or PD patch to receive MIDI/OSC messages from your input and forward them to Wekinator, just like the first Max/MSP example in this list.
    

Synthesis/Animation Examples & Instructions

Wekinator can be used to control ChucK, Max/MSP, PD, SuperCollider, Processing, Java, C++, Unity3D, or just about anything else that can receive OSC messages.

Wekinator comes packaged with a set of ChucK synthesis patches that you can use without any coding. These are located in the Wekinator directory, in project/chuck/synths/. To use one of these, open Wekinator, click "Edit chuck configuration" and a window will pop up. Then, in the "synthesis" tab, select "use a chuck synth class" button, then click "Choose file...” and choose your synthesis class! (fm_synthesis.ck is a good one to start with.)

To use Wekinator to control another environment, your code/environment should listen on port 12000 for incoming OSC messages with message string “/OSCSynth/params,” where each message contains a fixed-length list of parameters sent as floats. Also, in order to set up Wekinator to send parameter values to your synth, edit the ChucK configuration upon opening the Wekinator. In the “Synthesis (output)” tab, select the option to control a Max/OSC synth module, and configure this module using the "Configure" button. Add the proper number and types of parameters to control your synthesis module. Wekinator will create a classifier for every discrete parameter and a neural network for every non-discrete parameter. For discrete parameters, also set an integer value for the maximum value that parameter can take (if m is the max value, Wekinator will allow m+1 classes, labeled 0, 1, ... m). Leave “Distribution?” unchecked for now.

The workshop example bundle contains the following non-ChucK synthesis/animation/etc. examples:

1. •Max/MSP: simpleSine.maxpat takes 2 parameters: 1 for pitch (0 = middle C, 1 = C#, etc.), 1 for gain (0 to 127)
   

2. •Max/MSP: blotar.maxpat takes 9 real-valued control parameters; requires you to install the blotar object from PeRColate.
   

3. •SuperCollider: superColliderListener.rtf contains code that will set frequency of a SinOsc to Wekinator’s 1 real-valued output. See comments in code for instructions.
   

4. •Processing: The folder processingAnimations/ contains 3 simple animation examples that can be controlled by Wekinator:
   processing_continuous_hue: Controls hue as a single continuous parameter. Range is 0-255.
   processing_discrete_hue: Controls hue as a single discrete parameter, where 0=red, 1=green, 2=blue
   processing_continuous_hue_and_position: Controls hue and x- and y-positions
   

Advanced Techniques

Want to do more?

1. •Your custom OSC feature extractor can also optionally send an OSC message containing the feature names, so that these names are displayed in the Wekinator GUI. This message uses the string "/oscCustomFeaturesNames" and contains a list of string names (one per feature, in the same order as the feature values are passed in the oscCustomFeatures message). Send this message sometime after Wekinator is launched.
   

2. •Your non-Chuck synthesis environment can also optionally send parameter update messages to Wekinator. For example, if you have a sine in a Max patch whose frequency can be controlled either by Wekinator or by a slider, you can route the slider outputs to update the Wekinator parameter values in the GUI. See
   

3. •You can also implement a custom feature extractor directly in ChucK and not worry about sending OSC. See instructions here.
   

4. •You can implement a custom synthesis class directly in ChucK and not worry about receiving OSC. See instructions here.