Pcomp Assignment 2 (updated week 8)

Week 7 start:

I’ve always been interested in exploring the boundary between human subjective creativity and artificial, machine based generation. With the popularization of computer assisted creative software and machine learning technology, more artworks today are being created either with the help of various software, or fully generated using an algorithm.

After learning about serial communication between p5.js and Arduino, I was inspired to create an interface which allows the user to create abstract paintings on P5. Specifically, I took reference from the artist Jackson Pollock, whose works are known for expressing the artist’s subconscious using randomly placed strokes and splatters on a canvas.

I am hoping to create an interaction which takes in analog input from Arduino to be used as data to create abstract compositions similar to that of Pollocks on P5. I was able to successfully input signal from a joystick module in this week’s pcomp lab, by modifying the P5 code, it will be possible to make a painting on P5 using a joystick module on an Arduino.

I have written a test sketch for the project on P5 so far which creates dots and lines based on mouse position: https://editor.p5js.org/ukpfilms/sketches/_mARZkCcG

I also made another sketch featuring an interactive shape building based on webcam input and mouse position which could also potentially work well for this idea: https://editor.p5js.org/ukpfilms/sketches/0Ia0ZObuj

trippy mirror preview from Jack on Vimeo.

I will continue experimenting with the P5 sketch to add functions and visual elements to improve the end product.

Week 8 Update:

Since last week, I have modified the P5 sketch and made plans for prototyping/housing for the interface.

I combined elements from the two previous ideation P5 sketches and made a new sketch which draws pixels on the canvas based on camera input. The pixels are drawn based on the brightness of the camera input so the user can control where the pixels are drawn by moving around. In the current sketch, the colors of the pixels are controlled by the mouse position and clicking of the mouse changes the shape of the pixels between circle and square. In the final iteration the control input can be replaced by a joystick module on an Arduino so users can change the color with the joystick while they draw with their body posture, similar to how a painter would draw with a brush on one hand and hold the palette with the other hand when painting.

P5 sketch: https://editor.p5js.org/ukpfilms/present/f3co89EaQ

(only works on computers with a webcam)

Here are some digital paintings I made with the sketch:

Mirror Canvas Preview from Jack on Vimeo.

In terms of housing, the current plan I have is to use a sketch similar to the current one with the mouse input replaced with joystick input. The sketch will run on a laptop with a webcam and the graphics will be projected onto a backdrop in a room. The audience will be able to walk into the room and change the graphics on the projector by moving around while changing the color of the pixels with the joystick, painting a digital abstract painting through movement.

I experimented with extending the joystick module with a tennis ball, which would allow the audience to interact with the module with their palm instead of finger. I will decide later if I want to use the tennis ball extension, use the default thumbstick, or explore other ways to modify the joystick.

Week 9 update:

Painting with my own image from the webcam input while changing color was interesting in the previous P5 sketch, but it was restricting that the outcome of the painting was determined by the user’s silhouette and other shapes in the environment.

To make the interaction more resemble painting freely using a brush, I decided to change the P5 sketch so a shape is only drawn when a pixel from the webcam reach a certain brightness, which allows one to use a flashlight to paint where ever desired in a dark environment.

A feedback which I received in the class discussion last week was that users will not know to use the joystick module to change the color when drawing. I attempted to address this by housing the joystick inside a box with a diagram that hints at the color control direction orientation of the joystick.

The joystick connects to the Arduino which has serial communication with the P5 sketch running on the laptop. The laptop display is projected onto the wall using a projector.

I setup the project at my photo studio and had a chance to let some of my friends and family try out the interaction:

Digital Pollock from Jack on Vimeo.

Some notes for improvements based on watching people interacting with the project:

-webcam detecting light is not the most reliable input to use, the bright area of the flashlight always changes to results in uncontrollable random changes in the brush size in the P5 sketch.

-joystick is not the most intuitive way of controlling colors, despite the label on the controller, people didn’t understand how to get the color they want and often forget to change the color because they are too busy & confused figuring out how to draw with the flashlight