Quite a few Processing users (incl. myself) are working professionally and/or experimentally with (data) visualizations. Interesting and good works in this field are not just down to ingenuity of their authors but also largely dependent on quality data sources. Often those can be quite hard to come by, especially if you're reliant on free data sources. The creation and preparation of your own data can turn into a big stumbling stone since you are suddenly confronted with major technical issues about retrieval, parsing, storage, transformation, putting bits of data into relationships etc. No wonder a lot of amateur experiments are based around the readily available data as, for example, provided by Flickr, Technorati or del.icio.us.

As the blogging and Open Source movement has shown, innovation in any domain can and does happen bottom-up and amateurs play a major role in that. As Paul Graham writes:

There's a name for people who work for the love of it: amateurs. The word now has such bad connotations that we forget its etymology, though it's staring us in the face. "Amateur" was originally rather a complimentary word. But the thing to be in the twentieth century was professional, which amateurs, by definition, are not.

That's why the business world was so surprised by one lesson from open source: that people working for love often surpass those working for money.

Related anectode about amateur hardship: When working on base26 two years ago, I've spent about 5 long nights going page by page through the Oxford Dictionary manually filtering four-letter words and noting down their usage types, all for lack of an electronic version with this information.

On a large scale, good quality Open data is still generally rare, but steadily growing across various domains. The success of XML based standard data formats like RSS has been playing another important role on the road to liberated and readily usable data, yet the inherent problem with these formats is their lack of (direct) support for multi-dimensional and multi-directional data relationships.

So with these (amongst many other things) in mind, today reasearchers at Austrian company System One have announced the release of a snapshot of the entire English version of Wikipedia, converted into common flavours of RDF (RDF/XML, Ntriples and Turtle) and licensed as GFDL. Wikipedia³ , a monthly updated dataset currently counts in at approx. 47 Million triples (metadata statements about wikipedia articles) and that so far only includes the combined structural information of each article, like internal link and category relationships. A separate dataset containing the actual annotated articles is planned as is support for inter-wiki relationships and a SPARQL interface for processing remote queries.

This is a pretty amazing endeavour and hopefully will provide enough incentive for people to pick up and learn to use RDF technologies as flexible and powerful tool also for infoviz purposes.

Based on yet another superb idea by Mr. Tom Carden, we launched a new community site/wiki into the Processing universe today:

processinghacks.com

We're still in the process of collecting, adding, updating and heavily editing articles, but hoping to turn this wiki - with your help - into something worthwhile, stable and useful, above all! The wiki itself is a customized and extended version of the great DokuWiki platform with added features especially for Processing (syntax colouring, link types, applet support (coming real soon) etc.)

This is just a quick heads up before the weekend starts, everything is much better explained on the site...

As a follow up to yesterday's stormy discussions on the Processing discourse, I have to say I've been quite perplexed by the impact of my last post and the intensity of the discussion started. It's true I got things mushed up got carried away with some of my subjective views, yet I think Processing can't be talked about merely as tool. We all know that and I take it the resulting intensity is a sign we all feel Processing has become a very important part of our (at least) creative lives...

The 2 main points of criticism of my raised issues were:

"I also think it encouraged a slightly superficial view of computational design by quickly gaining cult status amongst people never been exposed to programming before. I think it's dangerous and a sign of crisis if every recycled L-System, Neural Network, Wolfram automata or webcam tracking experiment automatically is considered art (by their authors), simply because it's been "(Re)Built with Processing"..."

For some reason (Sorry, I can't quite isolate it) this has been interpreted as me preaching elitism or belittling beginners which is absolutely not the case. As I mentioned previously, my entire knowledge of programming is based on years of playful (often seemingly meaningless) experimentation. The above statement was more concerned with some of the emerging ego issues amongst the userbase. I think I also finally realised (not for the 1st time) that my concept of an artist is strangely incompatible with that of most others and my remarks where caused by this.

FWIW I consider an artist to be a seeker of knowledge, understanding, a master of craft and resulting insights into the very nature of what we call the universe. This is a process which doesn't require nor has it place for bullshit. Life's too short for such things and I can't grasp why people do it instead of trying harder and raising their own standards a little. My mummy told me, be quiet if you don't have something useful to say.

Maybe I should have taken this partial advice yesterday, yet I think the gist of my argument is valid somewhat, but should have been expressed differently. Don't be hurt!

"In terms of pure expressiveness of ideas, concepts and thought processes as code, Processing is inferior to straight Java or dynamically typed languages like JavaScript or Ruby."

Now this really was a tricky one and definitely should have been expounded by me a bit deeper. What constitutes Java is not just its overblown and almost uncomprehensible standard library, it also is just a language with very nice features. Try to consider them on their own, the statement above was referrig purely to that. The same goes for the other two languages mentioned. I know exactly where most of you were coming from with your criticism of this above statement: Getting simple things done in straight Java can be cumbersome. On the other hand the language has a handful of simple mechanisms to allow us to construct increasingly complex systems of code without having to constantly reinvent the wheel. My question to that was why does "Processing" not make use of those handy features? - and here it's not really the tool I am talking about but the way those things are totally excluded from the processing reference. If people (beginners) are never made aware that things exists how can they learn about it (Google doesn't count here!)? I also think the teachers amongst us should maybe give students a little bit more credit in grasping OOP concepts?

Another weird question, but it would be ++interesting to know: Has it maybe to do with the fact that most of you (teachers) have never been taught OOP/design patterns yourself?

A while ago I decided for myself that everything (code related) I do should be specialized and yet generic enough to be reusable in future projects, but alas I cannot do that fully with Processing in its current form.

Re: JavaScript and other dynamic languages. I think Ben's argument of saying he can't produce his work with JS is a slight paradox. If he would have chosen JS as base language for his project (with an underlying layer in C, e.g. by using SpiderMonkey) - he maybe would be able by now to do so. IMHO it's a little bit misconstrued, but I won't hit on it any further, since as he rightfully pointed out, there're much more important things to be solved at current.

Reason why I mentioned JS (and above all Ruby) as alternative was because those dynamically typed languages allow for very powerful coding idioms which one can only dream of in Java. But of course these again are very advanced topics and as such are only interesting to that part of our community. This was one of the mushed points of my last post.

Keeping on talking about object orientation just a bit longer, I find it very interesting Casey mentioning MAX, PD and VVVV as tools being almost purely targeted at the artist community. All of those tools are not dealing with text based programming, yet are deeply rooted in an object oriented approach. In fact their entire essence is based on an huge number of tiny encapsulated objects, fulfilling very specialized tasks and are loosely coupled. In order to create a "sketch" people choose them like building blocks and connect them via "wires" to create a directed process graph/execution flow.

So I think some take away points from that are:

• Objects are not alien concepts to programming newbies - I think it's all down to teaching styles. The visual programming metaphor obviously helps beginners on those platforms more since with the textual approach of Processing the connecting wires are initially invisible and have to be mapped out and "stored" in the students mind.


• Secondly, the highly modular architecture of those systems has contributed to their incredible success (IIRC, MAX has been on the market for +15 years). Also because of their fine grained modules (which have to be written in C++, lo and behold...) beginners can start out writing just tiny atoms of code instead of thinking on library scale. Because those tools enforce by their nature all objects/patches to be encapsulated, every effort spent on writing those extra bits of code is well spent in the knowledge it can be easily reused, potentially in infinite ways. This behaviour is not the case with Processing.

To back up some of the above, here're some search results for the word "object" for the various community websites:

PureData ~35200 results
VVVV ~2400
Processing 442 results

I couldn't find a decent start site for MAX, but there's maxobjects.com which is hosting roughly 2900 MAX/MSP objects of various complexity.

Do you get my drift?!

Note: This article is using "computational art/design" as compound term also including "generative" approaches.

Code lies at the very heart of computational design, a discipline becoming increasingly popular as proven by the mushrooming number of blogs, books written, conferences and workshops held - all using "code" as their core concept and pivotal sales hook. Yet there's apparently little intelligent discussion taking place about Code (with capital C) which goes beyond the art theoretical/cultural mindset and touches more on its raw (dare I say "technical" side), discussion about its manifold structures, expressiveness, metaphors in generative systems...

Code is language. It can be used and articulated in infinite ways. Poets use language in different, often far more subtle and sophisticated ways compared to our average modes of conversation. Their mastery and/or unique approach to language is what makes them artists, even before theorists can utter the words "political" or "historical" as their main point of interest. I much rather subscribe to something like Tolstoy's naturalistic description of art.

So personally and especially in regards to computational art, I find myself repeatedly standing in direct conflict with the often voiced opinion that literacy in the digital medium is unobtainable or even undesirable.

Processing...

...is a real phenomenon. Heralded as the new "it" tool for computational artists, it actually doesn't directly embrace or promote any state-of-the-art software designs (i.e. code structures). It's true, Processing has been primarily developed as teaching tool and always had in mind a beginner target audience, yet I've been thinking for quite some time that it merely delays the learning curve and lures in an increasing number of users (or shall I say "ongoing generative artists"?) with its easy to learn (and teach!) syntax to get quick visual (mainly) bang-for-the-buck. There's no arguing about its potential as digital sketching tool and its suitability for short workshops.

Being focused on small code sketches/experiments and used by various respected artists the tool created an huge amount of interest fairly quickly. In retrospect (well, for me after almost 3 years) I also think it encouraged a slightly superficial view of computational design by quickly gaining cult status amongst people never been exposed to programming before. I think it's dangerous and a sign of crisis if every recycled L-System, Neural Network, Wolfram automata or webcam tracking experiment automatically is considered art (by their authors), simply because it's been "(Re)Built with Processing"... Of course this is in no way to attack the tool or its intent, but is my growing issue with the surrounding community ethos. We have blogs writing about data equals nature and math being the language of nature, yet there doesn't seem to be any deep understanding of the importance of clean code designs and intelligent data structures or even community interest in further researching and experimenting with those artistically.

boolean isWrong = ( isExperimental != hasGoodDesign);

In fact from conversations with various fellow Processing users and lecturers I gather most are not aware of the total absence of decent software designs in the majority of the work produced with the tool so far. Due to the simplicity of its syntax, authoring environment and reference examples, the implicitly encouraged coding style is somewhere between procedural C programming (minus the pointer mess) and barely scratching the surface of object oriented designs.

In terms of pure expressiveness of ideas, concepts and thought processes as code, Processing is inferior to straight Java or dynamically typed languages like JavaScript or Ruby. Its ease of use has been gained by sacrifying scalability. Processing is based on convenience methods and it shows indirectly too. The Processing community at large has started to grow into one of consumers of previously written code.

On the other hand I believe artists (ongoing or not) working with "computational strategies" (can we please quit the marketing speak) must, or at least should, be aware of and work on intelligent software designs in order to advance the(ir) discipline. Form follows function.

In response to that I also believe it might hurt Processing as platform in future if experienced users will find themselves forced to breakout and leave the tool behind. To pre-empt this to happen, I think the community at large should pay more attention and spend time on extending the current library base. Above all, library authors should also respect the tremendous amount of work put in by Ben+Casey so far and too embrace the open source mentality of their core tool. The licenses are many (as well as much misunderstood, choose wisely!). That way existing library functionality can be further extended without having to reinvent the wheel (yet again!)...

An extensive library base for Processing will help the tool's longevity, even if users will slowly outgrow the initial proposal of the tool and only continue to use it library itself.

Open source is for doers. Happy, belated 2006! Glad I'm still alive...

I'd originally posted a follow up about using jEdit as editor here half an hour ago, but then quickly had to remove it again, since Blogger had to be helpful(?), automatically "clean up" my tables and was totally breaking the layout.

So in all briefness, let me re-direct you to its new home for some serious editing goodness. You won't regret it... ;)

Usually I don't do link blogging, but there're exceptions to every rule.

1st Ave Machine is a CG animation/postproduction house in NYC, doing some very slick projects mixing (semi)generative 3d and real world footage to blur the line between "what we perceive as real and impossible". Mutated botanics, weird worm-like creatures, glossy urchins, near perfect light matching, shaky cam, added film grain - the works. It's a model citizen of Uncanny Valley. A very inspiring one, I think...

Now if only that'd be interactive, some of the creatures would make some cute cyber pets ;)

Recently I've been receiving quite a few emails with an empty body and the subject line "tell me more!" - my standard one used for embedded mailto: links in early Flash experiments I'd published on various sites. I had hoped all these sources of neglect would have been eradicated in meantime. Now thanks to the 1st person in years to actually include a message in of these emails the culprit turned out to be my 3D DotMatrix Scroller. Done back in 1999 with Flash 4 it somehow seems to enjoy some belated attention again.

Oh my... The doors of perception opened wide and all the memories came flooding back at once. The mental pain of programming by selecting commands from a drop-down list and filling in form fields. Not being able to define functions, no local variables, the lack of most mathematic functions and operations apart from the main four. Flash forced one to adapt a surprisingly low level programming methodology, alien to many if not to most of its users, but especially the ones then considered primary target group: (web)designers. At the same time, Macromedia's chosen approach to get the beginnings of ActionScript into their tool was quite cunning. Those with a keen interest in interactivity and those who remained stoic in the light of these severe limitations of functionality (and usability!) came up with many creative solutions - sometimes by re-inventing the wheel, sometimes with true original thought. It very much reminded me of my previous dealings with writing applications and games in assembler (or before that in straight hex code). It too was "Xtreme Programming" with a much more literal meaning of the term than it is used currently. Many of us have a story to tell about the Oldskool ways of writing software. Remembering these things sometimes is important to me, as it's not always the very opposite to the way software is being built today. Often we're still being forced to reinvent the wheel, because we rely (too much?) on many other tools created without a long term vision. Looking back sometimes helps us to see things more in perspective.

Of course the lack of essential coding concepts in a tool like Flash was questionable, though limitation is not always a bad thing. I'm certainly not the first one to stress its benefits when applied to creative processes (where I consider coding to be one of them). Yet, in our age of apparently infinite possibilities and constant exposure to tempting mental side tracks, interruptions, consciously choosing limiting factors for one's work requires growing mental strength, focus and effort. This is especially true when we're repeatedly told only being limited by our own creativity (or the lack of it).

Once again less is more. Limit the noise. Often we forget limitation (consciously or not) can nurture and encourage experimentation, improvisation and hence creative thinking. It also can help you focusing more on core ideas of a (new) project and lead to a deeper understanding of a concept or an entire medium than any textbook can offer. I'm sure all those factors were a major influence on the simplicity and resulting popularity of Processing.

I always believed in and practiced "learning by doing". So far I've been challenged the most and learned the most in situations ruled by limitation: lack of time (both in project duration and execution time), available tools, assets, grand ideas, and above all lack of prior knowledge about best practices. However I attempt to take away exactly the latter from each project I've worked on. You could call it learning the Hard Way and sometimes it involves scaling down my own expectations prior to starting work in order to tackle the core parts of a problem successfully.

Though, what does "successful" mean in this case? Trying to be realistic about what you can do and what you can't is a two sided sword. It doesn't mean you should be happy with achieving mediocrity, relative to your true skills. It's more about figuring out and appreciating where you're on your path to ??? (fill in yourself) and then adjusting your goals.

The key is to accept, respect and admit failure. Personally, I think there's a big, positive mental payoff for a successfully completed project. The payoff is even bigger if you've managed to learn something totally new on the way, if your goal was beyond what you'd thought you could achieve - if you have succeeded in the face of limitation and superseded yourself. But even a failure is another piece in the big puzzle and is always part of the equation, at least to an extend.

No big punchline here. A personal era is coming to a close and it was time to reconsider...

just a quick note to announce a minor bug fix in the screen scraped Atom feed of the Processing discourse: the RegExp to isolate single posts was a bit dodgy and the 1st paragraph of each went AWOL on the way. no more so (at least until someone updates the HTML of that page... ;)

btw. the link to the personal profile page of the author of each post is fixed now too. thanks for your attention, please continue with your regular online activities!

Inspired by Jesse Andrews' idea to generate Sparklines with JavaScript, I just spent the last 3 hours writing an alternative version to use bitmaps instead of SVG as platform for the visualization. Apart from the limitation of only working in B&W, the XBM image format is highly suitable for this purpose, as the bitmap data can be manipulated with JavaScript. (btw. Wolfenstein 5K must be the by far most impressive use of this format ;)

Krister Olsson has just posted the first release of his new sound library Ess, making use of the JavaSound API, which is part of Java since v1.3. That is really good news as that way the library can be easily used in conjunction with other java sound libs, like the Tritonus API plugins or JLayer. It also doesn't require people to have the JSyn browser plugin (although requires the Java plugin, but that's less of a problem IMO).

The library is also a lot more feature rich than Sonia currently is, which until now has been the de-facto standard lib for using audio in Processing. Ess, on the other hand, has high-level synthesis functions for basic waveforms and various types of noise. It also has filters and analysis tools (FFT code still apparently based on my own hacks ;)

Looking forward to play around with it more, just hope Krister will release the source code too, and support the system's AudioInputStream...

short notice as usual... This year's Cybersonica audiovisual culture festival will take place this weekend (28/04 - 01/05) in London, hosted at illustrious venues like the Dana Centre (Science Museum), ICA and Truman Brewery in Brick Lane.

Via Stanza, initiator and curator of SoundToys.net, I just received final confirmation about Thursday's opening programme. As part of the symposium taking place at the Dana Centre in the afternoon, I shall be giving a brief presentation and overview of my sound-reactive works, followed later by a live performance with übercool audio artist pelado in the evening (1900-2230). Other artists taking part in the symposium include the fabulous Julian baker, up-and-coming pixelsumo artist and Processing fellow Chris O'Shea, Adam Hoyle and of course Stanza.

The festival's keynote speech will be delivered by none else than the almighty Golan Levin - definitely looking forward to this one!

If you're in town, let's meet up!

ps. phew, that was enough name dropping for a whole year! :)

WOW! i guess this is one of the days Ben and Casey have been dreaming about and (even more intensely) working towards for the past 3+ years. a big congratulations to them and all involved for reaching BETA!

good things come to those who wait...