Flash Posts

Try { Harder } 2012 Levelup

Last year I was fortunate to be one of the lucky 16 to attend the first try { harder } experimental conference / coder retreat in Nottingham. As mentioned in my blog post at the time it was an incredible 4-day event in which I learnt so much from some of the most talented and experienced people in the flash community.

The event was such a success that not only is it going to run again this year (which I hope very much to attend) but there is also an additional “levelup” conference happening next week. The idea behind the levelup is to provide businesses and freelancers with an intensive 4-day training and motoring session. The mentors are hand picked from the previous try harder members and so represent some of the best in their field.

What’s more Stray, the primary organiser of the event has just announced that Jetbrains the makers of the popular IntelliJ IDEA IDE are going to be sponsoring the event meaning each paying participant will receive a licence to use the IDE!

Interested? Head over to the levelup site and sign up: http://www.tryharder.org.uk/level-up-2012/

 

Why Developing for WebGL Sucks!

For some time now I have been working with WebGL and have developed a sort of love/hate relationship with it. I love the ability to instantly target millions of people with GPU accelerated code without any plugins or barriers (excluding the targets that dont support it). However as a developer, writing code that takes advantage of WebGL kinda sucks.

Procedural Based

First off is the way you have to structure your GL calls. For example take a look at the following generic bit of webGL harvested from the net:

[codesyntax lang=”javascript” lines=”normal”]

texture = gl.createTexture();
   gl.activeTexture(gl.TEXTURE0);
   gl.bindTexture(gl.TEXTURE_2D, texture);
   gl.pixelStorei(gl.UNPACK_ALIGNMENT, 1);
   gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, 64, 64, 0,
     gl.RGB, gl.FLOAT, new Float32Array(pix));
   gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
   gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

   texture1 = gl.createTexture();
   gl.activeTexture(gl.TEXTURE1);
   gl.bindTexture(gl.TEXTURE_2D, texture1);
   gl.pixelStorei(gl.UNPACK_ALIGNMENT, 1);
   gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, 64, 64, 0,
     gl.RGB, gl.FLOAT, new Float32Array(pix1));
   gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
   gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

   FBO = gl.createFramebuffer();
   gl.bindFramebuffer(gl.FRAMEBUFFER, FBO);
   gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0,
       gl.TEXTURE_2D, texture, 0);
   FBO1 = gl.createFramebuffer();
   gl.bindFramebuffer(gl.FRAMEBUFFER, FBO1);
   gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0,
       gl.TEXTURE_2D, texture1, 0);
   if( gl.checkFramebufferStatus(gl.FRAMEBUFFER) != gl.FRAMEBUFFER_COMPLETE)
     alert(err + "FLOAT as the color attachment to an FBO");

[/codesyntax]

All it does is create a couple of textures, setting their starting values and creates two frame buffers for rendering to. Its just it looks complicated and difficult to understand.

GL works on a procedural basis, so you tell GL that you are about to work on something by calling a function like “bindTexture()” then on the next line you perform an operation on it such as “pixelStorei()”. Now this may have made perfect sense back when we were writing everything in C which is procedural anyway however this is Javascript (or haXe in my case) which is an Object based language, code like this is difficult to understand and follow.

The procedural nature of WebGL means you have to be much more careful about unsetting things you have previously set. For example if you bind a texture to perform some operation, you must then remember to unbind it else you could inadvertently cause operations to be applied to it on subsequent calls elsewhere in your codebase. It this ‘hidden state’ that has caused me alot of headaches when developing my samples.

The principal behind WebGL was to provide a very low-level library which other developers can build upon to build more complex and abstracted libraries. And there are numerous libraries out there. I personally have tried several of them including the very popular three.js. Three.js is great for doing common things like loading models and putting quads on the screen. I however encountered a problem with render targets which I struggled with for days before I discovered that you had to set “needsUpdate” to true on your texture before using it. In the end I decided to drop three.js beacuse of another issue I encountered and instead attempt to reduce my complications by working with webGL directly.

Flash11’s Stage3D has the same philosophy as webGL, to provide a low level API for other developers to build libraries upon. The thing is Flash11’s low-level API makes more sense and is more readable. For example the following to me is much more readable than its webGL equivalent:

[codesyntax lang=”actionscript3″ lines=”normal”]

texture = c.createTexture(logo.width, logo.height, Context3DTextureFormat.BGRA, false);
texture.uploadFromBitmapData(logo);

[/codesyntax]

The Stage3D API also uses language like “upload” to let you know when you are transferring data to the GPU, for a new comer to GL you have no clue when things are going to the GPU. Its small things like this that reduce the “WTF?” factor when tackling the tricky world of hardware-accelerated 3D programming.

Cross-domain textures

This one cropped up around July time this year and took me ages to work out what was going on. For some inexplicable reason (or so it seemed) my code one day stopped working. When I looked for demo code online it all worked fine, however when I downloaded it and run it locally it also didnt work. I was getting errors like the following:

Uncaught Error: SECURITY_ERR: DOM Exception 18
Uncaught Error: SECURITY_ERR: DOM Exception 18
Uncaught Error: SECURITY_ERR: DOM Exception 18
Uncaught Error: SECURITY_ERR: DOM Exception 18
Uncaught Error: SECURITY_ERR: DOM Exception 18 

I was so baffled that I posted about it on the HaXe mailing list asking for help, thinking it was something I was doing wrong with HaXe. It turns out (after much wall-head-butting) this was a change that they brought into Chrome 13 and Firefox 5 to combat a security problem when using shaders that use textures from a different domain to the one running the code:

http://blog.chromium.org/2011/07/using-cross-domain-images-in-webgl-and.html

Now I have no problem with cross-domain issues, im used to this from Flash where we have the same sort of setPixel() restrictions on cross-domain BitmapData’s. The thing is, it appears that this restriction applies when running code locally too. So If you are developing something on your local machine and trying to access a texture from disk it throws the same security errors because the browser thinks you are reaching across domains to access the image.

At the time the only way to get around this was to create your own webserver that you run on localhost to server up the files. So to do that I had to download python so I could run a simple localhost commandline webserver from my bin directory. What an effort! There may be easier ways these days to solve it but at the time it really frustrated me and formed yet another barrier to developing webGL.

No Error Messages

This is by far the most annoying thing about developing for WebGL. So many times I have been trying to write something that I know SHOULD work but for some reason it doesn’t. I dont get any error messages, nothing. It makes writing something from scratch neigh on impossible.

In my last post “GPU State Preserving Particle Systems with WebGL & HaXe” I started with an idea. I attempted to code it ‘bottom-up’. That is start with nothing and then add more code until I reached what I wanted. Unfortunately having no error messages in WebGL makes this very difficult indeed. I would spend some time writing something really simple, like trying to get a textured quad to render on the screen only to find I get nothing. I double check my camera matrices my vertex and texture buffers, my shader and still nothing. Eventually I found that I hadn’t bound something first before trying to operate on it *sigh*

In the end I found the best way to get anywhere is to go from the other direction, a ‘top-down’ method. Start with something kind of simmilar to what you want then cut bits out one line at a time until you get what you want. Its extremely time consuming and frustrating, but its less frustrating than going from the other way.

There are tools out there that help with debugging what is going wrong. Namely the WebGL Inspector (see above) is intended to provide gDEBugger / PIX like debugging information about what is going on inside webGL. Its a clever bit of tech, it lets you inspect buffers and traces each gl call, however it suffers from the same underlying problem of having no errors. You setup a buffer incorrectly and what you get is “INVALID_VALUE”. No indication as to which of the values is invalid or what part of the call you messed up on :(

Googling Doesn’t Help

If you do happen to get an error message (unlikely) or you word your problem in a sufficiently succinct and googaleble way you will then run into the next big problem with WebGL; theres very few people using it. Now I know I am likely to be flamed for that comment, but it just seems that way to me. Whenever I tried to google my problem, or google for what I was trying to achieve (because working bottom-up doesnt work) there would be a very sparse smattering of relevant posts. Usually the results are forum posts and are OpenGL not WebGL related and are from 5-10 years ago.

But..

Now having just ranted on for several hundred words about why it sucks im going to finish it off by saying that im going to continue to develop for WebGL using haXe regardless. Why? Well I just like making pretty things that run fast and GPGPU programming appeals to me for some unknown (likely sadistic) reason.

Conway’s Game of Life in haXe [NME & MassiveUnit]

The second day of try{harder} was dedicated to a single topic; test driven development (TDD).

The group was split into pairs and given the task of using TDD to write a solver for the game of life in AS3. After an hour we then threw away everything we had done, swapped partners and repeated the process.

This was extremely valuable for me as I had never written a unit test before. Seeing how different people tackled the same problem was fascinating and informative.

After repeating the process three times Stray asked if I was interested in teaming up with another attendee of the conference Alec McEachran to investigate unit testing in haXe. It was a great idea as it meant we both could investigate how unit testing worked in haXe and it would give me another code example for my talk the following day.

After a brief search we decided on Mike Stead’s MassiveUnit for testing as the testing syntax looked similar to FlexUnit and it contained a toolchain for running the tests on multiple platforms.

An example of a test we wrote is:

[codesyntax lang=”actionscript3″ lines=”normal”]

package ;
import massive.munit.Assert;
import Grid;

/**
 * ...
 * @author MikeC & Alec McEachran
 */

class GridTest
{
	public var grid : Grid;

	@Before
	public function before():Void
	{
		grid = new Grid(3, 3);
	}

	@After
	public function after():Void
	{
		grid = null;
	}

	@Test
	public function initiallyThereAreNoLiveNeighbors():Void
	{
		var liveNeighbors = grid.getLiveNeighbors(1, 1);
		Assert.isTrue(liveNeighbors == 0);
	}

	@Test
	public function liveNeighborCountIsAccurate():Void
	{
		grid.set(0, 0, true);
		grid.set(1, 0, true);
		grid.set(2, 1, true);

		var liveNeighbors = grid.getLiveNeighbors(1, 1);
		Assert.isTrue(liveNeighbors == 3);
	}

}

[/codesyntax]

It should look fairly familiar to anyone who has used FlexUnit before. The metatags @Before @After and @Test perform in exactly the same way as they do in FlexUnit. Another benefit of using munit over the built in testing framework in haXe is that you are given a tool to run tests on all platforms simultaneously:

[codesyntax lang=”text”]

 haxelib run munit test test.hxml

[/codesyntax]

When executed you get something that looks like the following:

Which presents a nice graphical representation of the tests run and which failed (if any).

Once built and tested we decided to give the code a simple visual representation. We wanted to show off the ability for haXe to target multiple platforms. To do this we decided to go with NME which I had been experimenting around with recently.

NME is a library and tool chain for haXe designed to allow the developer to use the flash API on multiple platforms. They achieve this by writing platform targeted version of the flash API. So what this means is code such as the following:

[codesyntax lang=”actionscript3″ lines=”no”]

package ;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.display.MovieClip;
import flash.geom.Rectangle;

/**
 * ...
 * @author MikeC & Alec McEachran
 */

class Render
{

	private var _cellSize : Int;
	private var _renderTarget : BitmapData;
	private var _rect:Rectangle;

	public function new(container:MovieClip, cols:Int, rows:Int, cellSize:Int)
	{
		_cellSize = cellSize;
		_renderTarget = new BitmapData(cols * cellSize, rows * cellSize, false);
		container.addChild(new Bitmap(_renderTarget));

		_rect = new Rectangle(0, 0, _cellSize, _cellSize);
	}

	public inline function lock():Void
	{
		_renderTarget.lock();
		_renderTarget.fillRect(_renderTarget.rect, 0xff0000);
	}

	public inline function renderCell(x:Int, y:Int, isLive:Bool):Void
	{
		if (isLive)
		{
			_rect.x = x * _cellSize;
			_rect.y = y * _cellSize;
			_renderTarget.fillRect(_rect, 0);
		}
	}

	public inline function unlock():Void
	{
		_renderTarget.unlock();
	}

}

[/codesyntax]

Will compile down to flash, c++ and Javascript! NME also includes packaging abilities for webos, android and ios. So with a few scripted command lines you can target most app marketplaces:

[codesyntax lang=”text”]

haxelib run nme test YourProject.nmml flash
haxelib run nme update YourProject.nmml ios
haxelib run nme test YourProject.nmml webos
haxelib run nme test YourProject.nmml android
haxelib run nme test YourProject.nmml cpp
haxelib run nme test YourProject.nmml cpp -64

[/codesyntax]

What it means for this project is we could very quickly get a view for our game of life running in flash, JS and native desktop.

To show just how easy it is I made the following video:

You can see the HTML5 build here: http://mikecann.co.uk/projects/gameoflife/Export/html5/bin/

And the flash build here: http://mikecann.co.uk/projects/gameoflife/Export/flash/bin/MyApplication.swf

I have uploaded the source for the project here: http://mikecann.co.uk/projects/gameoflife/gameoflife.zip