I have just released cv.jit version 1.7.1, which fixes two issues. A bug in cv.jit.shift, which also affected cv.jit.touches, was causing memory leaks. Furthermore, some Windows users were seeing 14001 errors when trying to use externals that make use of OpenCV functions. Both problems have now been fixed.
Minor update to cv.jit
2010/08/16
Using cv.jit.touches: recycling index values
2010/06/29cv.jit.touches, a new object added in the 1.7.0 release, tracks the movement of bright spots in a greyscale image and reports “touch”, “drag” and “release” information – when regions appear, move or disappear. Each region is assigned a unique ID number, so that even if they move, you can always know where an object is from frame to frame. This is unlike using cv.jit.label-based blob analysis, where you are never guaranteed that the same object in a scene will end up with the same index.
By default, cv.jit.touches outputs ever-increasing indices, although the counter is always reset to 0 when there are no active regions. For some applications, this is a reasonable approach: every touch event has its own unique ID. However, some may wish to recycle IDs. If you’re in this situation, I made an abstraction that renumbers the IDs coming out of cv.jit.touches to re-use release IDs, and keep their values as low as possible. It will always output the smallest value that isn’t currently assigned to another active region.
Download cv.jit.touches.recycle.
In order to use this file, just copy it to the “Max5/Cycling ‘74/cv.jit-support/Abstractions” folder.
You can see how it works by copying the following code and selecting the “new from clipboard” option from Max’s file menu.
----------begin_max5_patcher----------
911.3oc0XssbhBDD8Y8qfhmcslaL.6a6Ovturus0VoPXhRJfwBFiIap7uuLW
zPRDbTPzTQfLyfNm9zce5FdY5D2E7mXUtNe24ONSl7xzISTSImXhY7D27nmh
yhpT2laLOOmUHbmoWSvdRnl+2qJYLmEY7EU6VqXSdZQFSn9hv2ljuQraVfY1
0Qh3UoEKuqjEKzvgR7lCl4.AgxK9pyH3bfyeMemzD0FyW7v27g61y64Ehhnb
lZoeTlFk0bkpz+oVAhmCjy95zoxSy5qsukOjVNwm1zxwcZ4fqpk+qBsOeXLb
LDYsgSCupF9OGTONBCs2vCFECOmUUEsj8ICONiEUNyYcTZgf+XTliGP9ABTG
MW.Jmt9.ATGebIpdEHrdkEQEKadd2tlkVvh4aJTaM4PTM5fTMrKYEcxEQGpE
n3ZJoEtldbtVushmWyzafq69eoAvMTv1VCjO4Ex3QIMIJKB+5fSLJs5rOIWz
QzG9LXDERGAVI9w4OjJlK3ahWwplmTFs8fDD4TIHOOUhIlpqBA75jhfmAEUi
66xiDkoOMnDkjNVuMsHgumIVWxEb4NuGYXDXwyHB3DBlPGkqH.UlU8ushqHf
VHKT64QyT4RWrviZH+bsZ1AsZ7AsZ7wiPvgZ0ktkvAmirxr8TxW2DIDJnQhD
zDbzBMAA2NIRGobnUEvPCRALCEZJfoaUD2R8qvqc4qqf9igeH.pM5OAim7ij
KJWt.ksIOZPMUDQ08nePmOnP+xkFZgm1xmVTyPN.4eybtW9+HOucGtCR9ioY
GS+e95QgsHTesyet.cgakTEdXnZC6piOgT0HRKbs2WUtdD09wfvlAuldNZQ7
mbi9rKRcvr3DGhS7pnRoBsyPK9iLuHAbm8WftsjDspWrSflvGmlBTYjAzN6B
6R2qp5dTYKe3EQp.tb92SZU7Mkw61IixgyaPOgUIRKhDo7hF2C5c2ypzjDVQ
yWJSdZxZdcBtABj.xbOYcAO4ED1e+nV5HvVrRFdrhgAJPBTw7FrpF0SrRGdr
1SDgr.Q9iJhBGdNpoGbP8m9VfU4yb4fOcv56qe72f8i5IXg1B1wyWKgDzFHA
GUHgrARnwCR.K7bAipiyp.+QEQTaBuonaNVRg6yPdfFJu36Q1OpuDH1FzNt9
zKP8RpmtADPPi9PTi5KXAVnbPgipxAEXg.KENpBrTfsYE2V0gniecHfMPBbS
UZj1mRi0Cdc5+MONptI
-----------end_max5_patcher-----------
cv.jit 1.7 is finally out!
2010/06/07The latest version of cv.jit, 1.7.0 is finally out. I say finally because, it’s been on the brim of release for several months now, but life being what it is, I only now managed to put the finishing touches on it.
The most obvious change is that the help files have been completely re-written in Max 5 format. cv.jit 1.6 and earlier help files did not display properly in Max 5, owing to some issue with double-byte comments.
A few objects have been added also. cv.jit.opticalflow combines the functionality of cv.jit.LKflow and cv.jit.HSflow. These were two of the earliest externals I wrote, and I now somewhat regret the decision to keep them separate: they essentially do the same thing, albeit in different ways. cv.jit.opticalflow also adds support for two newer optical flow estimation algorithms: block-matching and a brand-new bleeding-edge real-time algorithm by Gunnar Farnebäck. To go along with this new external (and the two older optical flow objects), I also added a drawing utility, cv.jit.flow.draw that displays the optical flow field using hue for direction and saturation for distance.
Farnebäck optical flow, visualized with cv.jit.flow.draw
I often get questions about tracking blobs, or about dealing with the fact that cv.jit.label doesn’t always give the same label to what we would perceive as being the same object. The new object cv.jit.touches, sorts of addresses these issues. It’s a greyscale/binary region tracker. It assumes that the regions are of roughly the same size and don’t overlap. As the name implies, it was packaged with multi-touch interfaces in mind, and it outputs information such as “touch”, “drag” and “release”, but it can be used with other kinds of inputs.
cv.jit.threshold implements adaptive thresholding, in which each pixel is compared to the average brightness of its surroundings instead of a fixed value. This is especially useful when working under slightly uneven lighting situations.
Finally, cv.jit.grab is a ridiculously simple but very useful abstraction that wraps jit.qt.grab and jit.dx.grab depending on your platform. This allows you to write cross-platform patches.
Another big change, apart from the help files, is that I moved the cv.jit site to my own domain. As mentioned in every help file and abstraction, IAMAS, the great media art institution in Gifu, Japan, has provided support for my work on cv.jit – in the form of computers, software, time, unwitting testers and advice from teachers and colleagues. Alas, my contract having reached its end, I don’t work there anymore and I thought that it might be best if I gathered all my work under the same roof, here. The actual files are hosted on Sourceforge, so that those who are interested in actually doing some development can dig in to the SVN repository.
Head over to the new cv.jit page for downloads!
Gragra DSP Exhibition 2010 – Kyoto
2010/03/22
Shinobu Toma – Ghost in the Space
Kazuomi Eshima – Remind
Kei Shiratori – twist suburbia
Daichi Misawa – Skies
Leo Kikuchi – Landscape in my Arms
Kaori Takemoto – Hunter-gatherer Colorist
Kanna Komaki – Utopian Babble
IAMAS 2010 Graduation Exhibition
2010/02/18
The IAMAS 2010 Graduation Exhibition opened this morning. Graduating students from the Academy and the Institute will be showing their works until Sunday February 21st.
After the traditional ribbon ceremony, a bus-full of high school students poured into Softopia’s Sophia Hall.
Mitsuru Tokisato – What Could Have or Can Happen?
Mitsuru Tokisato – What Could Have or Can Happen?
DSP course student Mitsuru Tokisato’s piece “What Could Have or Can Happen?” is a photographic record of his surrounding random objects with white tape.
Yuuya Ito – Cell #00
Yuuya Ito – Cell #00
Yuuya Ito, the other DSP student exhibiting, created and performed in a short play titled “Cell #0″. He is exhibiting the device he built for his performance: a large faucet that he plants in the sand. He is able to control video projections by turning the faucet and planting the pipe in various places.
Reinhard Gupfinger – Singing Robot Cricket
Reinhard Gupfinger – Singing Robot Cricket
Reinhard Gupfinger, an exchange student from the University of Art and Industrial Design Linz, in Austria built a robotic chirping cricket.
IAMAS DSP Course First-year Students Show their Works
2010/02/02Following last week’s graduation work presentation, first-year students of the Academy showed their end-of-year projects. Unfortunately, these won’t be at the upcoming graduation exhibition, but it will be possible to see some of them at a show we’re planning for next March.
Kei Shiratori – Isolated Island
Kei Shiratori – Isolated Island
Kei Shiratori presented an iPhone multiplayer musical game. Players, using the accelerometer, throw around a ball to each other, which generates sound depending on their actions.
Kazuomi Eshima – Remind
Kazuomi Eshima – Remind
Guitarist Kazuo Eshima built two heavy-duty controllers: a minimalist knob device and a monome-like foot controller. Both are built like tanks.
Kim Jong-Un created a sound piece meant to be listened to while riding a bicycle. As you go faster, the sounds of the piece blend in with the sound of the wind in your ears.
Leo Kikuchi – Landscape in my Arms
Leo Kikuchi – Landscape in my Arms
In Leo Kikuchi’s video installation, you get to observe the world from the perspective of the author’s swinging arms, as he walks around the landscape.
Bak Young-Hyo – Cube
Bak Young-Hyo – Cube
This installation by Bak Young-Hyo is a prototype for a larger outdoor piece in which a giant cubic video balloon floats in the sky.
Daichi Misawa’s “Skies” is a web-based piece in which photographs he took at Oświęcim (Auschwitz) are modified to create endless variations, in a commentary on plurality and identity.
http://www.iamas.ac.jp/~d.misawa09/skies/
http://www.iamas.ac.jp/~d.misawa09/skies/
Shinobu Toma – Ghost in the Space
This sound installation by Shinobu Toma uses several iPhones in a pitch-dark room. A simple interface allows visitors to specify parameters of loop length and loop repetition. Sounds are simply recorded and played back a number of times, but unexpected sounds arise from the complicated feedback that occurs.
Kanna Komaki – Show-room
In this video piece, Kanna Komaki transforms an everyday space (a staircase at IAMAS) through the use of a large number of lightbulbs.
Kaori Takemoto’s “Hunter-Gatherer Colorist” is a portable device that uses a colour sensor to allow the user to create colour palettes by literally gathering colours from the external world.
IAMAS DSP Course Students Show their Graduation Work
2010/01/25Today and tomorrow, second-year Academy students show their graduation work to teachers and fellow students. The general public will get a chance to experience the works during the IAMAS 2010 exhibition, which will be held from February 18 to 21.
Hideyuki Oda wearing Hoonida-Kim's "C_"
Hoonida-Kim – "C_"
Yuuya Ito – "Cell #00"
Mitsuru Tokisato – "Surround" (Temporary title)
Mitsuru Tokisato – "Surround" (Temporary title)
Yutaka Kitamura – "Form. Design + Personal fabrication"
Yutaka Kitamura – "Form. Design + Personal fabrication"
A Ruby script for generating Jitter attributes
2010/01/03Writing your own Max or Jitter externals in C or C++ isn’t terribly hard, once you’ve wrapped your head around the API’s C approach to object oriented programming. However, it does involve a fair bit of boilerplate. This is especially true for adding attributes to an object – an triply so if this attribute has custom getter and setter methods.
The cv.jit collection now contains more than a few externals and I find myself spending more time trying to find ways to automate some of the repetitive tasks that are required for keeping it up to date. One of the tools I just made is a nifty Ruby script for automatically generating all the necessary attribute-related boilerplate. Simply invoke it at the command line with only a few arguments and it generates a .c file containing the necessary code. It parses the arguments “-c”, “-l”, “-f”, “-d”, “-s” and “-a” as “char”, “long”, “float32″, “float64″, “symbol and “atom” types. Numbers (if there are any) as the number of elements in a list. The arguments “-get” and “-set” specify that the attribute has a custom getter and setter, while “-clip” will add a filter to clip argument values. Any other argument is going to be parsed as the name of the attribute, unless it begins with a “-”, in which case, it’s interpreted as your external’s name (periods are automatically converted to underscores.)
For example:
ruby ./jitargs.rb -f -cv.jit.bigbrother foo
This generates a file “jitter_args.c” in the current directory that looks like this:
//setter/getter declarations
//attribute variables
float foo;
//setters/getters
//attribute registration
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset,"foo",_jit_sym_float32,
attrflags,(method)0L,(method)0L,calcoffset(t_cv_jit_bigbrother,foo));
jit_attr_addfilterset_clip(attr,0,1,TRUE,TRUE);
jit_class_addattr(_cv_jit_bigbrother_class,attr);
//attribute initialization
x->foo = 0;
//attribute variables
float foo;
//setters/getters
//attribute registration
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset,"foo",_jit_sym_float32,
attrflags,(method)0L,(method)0L,calcoffset(t_cv_jit_bigbrother,foo));
jit_attr_addfilterset_clip(attr,0,1,TRUE,TRUE);
jit_class_addattr(_cv_jit_bigbrother_class,attr);
//attribute initialization
x->foo = 0;
If you wish to add more attributes, just run the script again with different arguments, new code will be inserted in the appropriate place. For example, by running the following:
ruby ./jitargs.rb -a -get -set 2 -cv.jit.bigbrother bar
The file above is modified to:
//setter/getter declarations
t_jit_err cv_jit_bigbrother_set_bar(t_cv_jit_bigbrother *x, void *attr, long ac, t_atom *av);
t_jit_err cv_jit_bigbrother_get_bar(t_cv_jit_bigbrother *x, void *attr, long *ac, t_atom **av);
//attribute variables
long barcount;
t_atom bar[2];
float foo;
//setters/getters
t_jit_err cv_jit_bigbrother_set_bar(t_cv_jit_bigbrother *x, void *attr, long ac, t_atom *av){
if(ac < 2){
//Not enough parameters?
return JIT_ERR_NONE;
}
return JIT_ERR_NONE;
}
t_jit_err cv_jit_bigbrother_get_bar(t_cv_jit_bigbrother *x, void *attr, long *ac, t_atom **av){
int i;
if ((*ac)&&(*av)) {
//memory passed in, use it
} else {
//otherwise allocate memory
*ac = 2;
if (!(*av = jit_getbytes(sizeof(t_atom)*(*ac)))) {
*ac = 0;
return JIT_ERR_OUT_OF_MEM;
}
}
for(i=0;i<2;i++)av[i] = x->bar[i];
return JIT_ERR_NONE;
}
//attribute registration
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset_array, "bar", _jit_sym_atom, 2,
attrflags, (method)cv_jit_bigbrother_get_bar,(method)cv_jit_bigbrother_set_bar,
calcoffset(t_cv_jit_bigbrother, barcount),calcoffset(t_cv_jit_bigbrother,bar));
jit_class_addattr(_cv_jit_bigbrother_class,attr);
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset,"foo",_jit_sym_float32,
attrflags,(method)0L,(method)0L,calcoffset(t_cv_jit_bigbrother,foo));
jit_attr_addfilterset_clip(attr,0,1,TRUE,TRUE);
jit_class_addattr(_cv_jit_bigbrother_class,attr);
//attribute initialization
jit_atom_setlong(&x->bar[0],0);
jit_atom_setlong(&x->bar[1],0);
x->foo = 0;
t_jit_err cv_jit_bigbrother_set_bar(t_cv_jit_bigbrother *x, void *attr, long ac, t_atom *av);
t_jit_err cv_jit_bigbrother_get_bar(t_cv_jit_bigbrother *x, void *attr, long *ac, t_atom **av);
//attribute variables
long barcount;
t_atom bar[2];
float foo;
//setters/getters
t_jit_err cv_jit_bigbrother_set_bar(t_cv_jit_bigbrother *x, void *attr, long ac, t_atom *av){
if(ac < 2){
//Not enough parameters?
return JIT_ERR_NONE;
}
return JIT_ERR_NONE;
}
t_jit_err cv_jit_bigbrother_get_bar(t_cv_jit_bigbrother *x, void *attr, long *ac, t_atom **av){
int i;
if ((*ac)&&(*av)) {
//memory passed in, use it
} else {
//otherwise allocate memory
*ac = 2;
if (!(*av = jit_getbytes(sizeof(t_atom)*(*ac)))) {
*ac = 0;
return JIT_ERR_OUT_OF_MEM;
}
}
for(i=0;i<2;i++)av[i] = x->bar[i];
return JIT_ERR_NONE;
}
//attribute registration
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset_array, "bar", _jit_sym_atom, 2,
attrflags, (method)cv_jit_bigbrother_get_bar,(method)cv_jit_bigbrother_set_bar,
calcoffset(t_cv_jit_bigbrother, barcount),calcoffset(t_cv_jit_bigbrother,bar));
jit_class_addattr(_cv_jit_bigbrother_class,attr);
attr = (t_jit_object *)jit_object_new(_jit_sym_jit_attr_offset,"foo",_jit_sym_float32,
attrflags,(method)0L,(method)0L,calcoffset(t_cv_jit_bigbrother,foo));
jit_attr_addfilterset_clip(attr,0,1,TRUE,TRUE);
jit_class_addattr(_cv_jit_bigbrother_class,attr);
//attribute initialization
jit_atom_setlong(&x->bar[0],0);
jit_atom_setlong(&x->bar[1],0);
x->foo = 0;
All you need to do now is copy and paste the code at the appropriate places. Of course, if I was really crazy, I would write a script that parses and modifies the actual external source but I’ll leave that as an exercise for the reader.
Jane Rigler at IAMAS
2009/11/30Flutist and composer Jane Rigler visited us at IAMAS last friday for a lecture and performance of a few of her pieces for flute and live electronics. Jane talked about how traditional musical traditions of places like the Solomon Islands influenced her work as a composer and performer, and showed us some of the Max patches she uses for her performances. She also talked about Music Cre8tor, a system for allowing physically and cognitively challenged children to play music together.
Jane Rigler and I, providing the Japanese translation
Photographs by Yosuke Kawamura.
Landschaft 1.0
2009/08/18
This is a synthetic early Japanese summer soundscape. All the material used here was generated from a video of a cicada song-drenched brook that runs near my workplace.
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
