Databending in Java

The above image is the result of databending a snippet of a video. This was created by extracting each frame of the video, loading it into Audacity (a music editor) as sound, applying a normalize filter to it and then exporting it back to an image file. After that the resulting images where converted into a gif.

In order to save time I decided to try and see if I could do it programmatically. First the image is loaded and the pixel data is extracted as a byte array. The byte array then has a sound effect applied to it, treating each byte as a sample of sound. The resulting byte array is then applied to an image and saved to a file. You can then do this for each extracted frame of a video.

Code

Loading Pixel Data

BufferedImage img = null;
try {
    img = ImageIO.read(new File("Z:\\Pictures\\image.bmp"));
} catch (IOException e) {
	throw e;
}
byte[] pixels = (DataBufferByte)img.getRaster().getDataBuffer()).getData();

Convert Bytes to Number

short[] values = new short[pixels.length/2];
ByteBuffer bb = ByteBuffer.allocate(2);
for (int i = 0; i < pixels.length-1; i+=2) {
	bb.put(pixels[i]);
	bb.put(pixels[i+1]);
	values[i/2] = bb.getShort(0);
	bb.clear();
}

Effects

The Pitch Shift effect used the code from here. Although written in c# it was easily converted to java. The Normalize effect is created by finding the maximum value of the pixels and then multiplying each value by the target maximum divided by the maximum. (values[i] = (short)(values[i] * (double)targetMax/(double)max)). The echo was created by adding the value of the pixels multiplied by the decay, to the value of the pixels the delay amount further down the array. (values[i+delay] += (short)((float)values[i] * decay)) The delay amount can be calculated by multiplying the delay in milliseconds by the sample rate. 

Applying to Image and writing to file

for (int i = 0; i < values.length; i++) {
    byte[] arr = new byte[]{(byte)(((short)values[i]>>8)&0xFF),(byte)((short)values[i]&0xFF)};
    pixels[i*2] = arr[0];
	pixels[i*2+1] = arr[1];	
}

BufferedImage resultImg = new BufferedImage(img.getWidth(), img.getHeight(), img.getType());
resultImg.setData(Raster.createRaster(img.getSampleModel(), new DataBufferByte(pixels, pixels.length), new Point() ) );
try {
	ImageIO.write(resultImg, "bmp", new File(Z:\\Pictures\\output.bmp"));
} catch(Exception e) {
	throw e;
}